Archive

Posts Tagged ‘cow’

On Debugging Virtual Applications Part 4: The Case of the Missing Dump

September 27, 2016 Leave a comment

I’ve been blogging lately (well . . . not exactly lately) about debugging virtual applications that misbehave – whether by hanging, crashing, or spewing some strange error message. In discussing the use of tools, one question I often get is the relationship of attaching the debuggers to the virtual application inside or outside the virtual environment. With App-V 5, it only matters in certain circumstances at the user mode level as virtualized applications work from a state-separated registry and file system that leverage the native file system and registry.

When a developer is troubleshooting a virtual application that they have developed, or an IT Pro Debugger is attempting to reverse engineer an application issue, I usually recommend they attach to the virtual process as they would any other application at first. You may decide due to necessity or preference to attach a debugger or a tool to the virtual application process within the context of the virtual environment by launching it in the application’s “bubble.” You may want to be advised that once you do this, the debugging tool will also be running virtually and will behave accordingly.

I bring this up due to an interesting issue that was encountered by one of our partners a while back: There was an issue with a plug-in to Internet Explorer causing it to hang shortly after the BHO had been triggered while running in standard isolation (i.e. no Dynamic Virtualization involved.) Given that the individual had developed the code for the plug-in, they wanted to capture a full user mode memory dump of the instance of Internet Explorer that was running virtually. The issue could not be reproduced in the developer’s environment so there was the usual suspicion of something environmental being a factor. Since standard WER (Windows Error Reporting) is somewhat limited by default, the customer was leveraging ProcDump -h to capture a user mode memory dump of the IEXPLORE.EXE process.

Here’s the thing: while Procdump appeared to attach to and generate a dump successfully, there were no dumps to be found per the developer. Upon further inquiry, I found that the developer was doing the following:

  1. Using C: as a target. This is not good on many levels.
  2. Running ProcDump from a command prompt in the application’s (IEXPLORE.EXE) virtual environment.

While running it in the bubble was not necessarily a bad thing, I had the developer simply redirect the output to %TEMP% instead of C:. The dumps showed up as normal. When asked why C: mattered, I told him that due to coupling factors (specifying an unexcluded folder, VFS Write mode, running as an administrator, and launching in the bubble, the dump file was treated as application state data and was redirected to the VFS CoW location. Upon a quick demonstration, it was discovered buried beneath the user’s VFS folder.

 

164

 

 

I should note that there shouldn’t be any concern with regards to the usability of these redirected dumps. I will also note that upon repairing the App-V package in questions, the user freed up around 12GB of disk space. I guess they had been trying the ProcDump command quite a bit. J

 

App-V 5: On the App-V 5 Virtual Registry

April 16, 2015 1 comment

I have been meaning to follow-up my previous discussions of the App-V registry staging subsystem with an article on the virtual registry in App-V 5. I will admit I am a little late to the follow up, however, as they say “better late than never.” In previous article I discussed registry staging and its effect on publishing and streaming. Now the discussion continues with how the virtual registry handles operations once the virtual registry has been staged.

The Virtual Registry in App-V was implemented in a much more streamlined way in version 5 than in previous versions. First of all, real hive files are packaged with the APPV package format. In addition, the real registry is used where the actual locations are state-separated while the Virtual Registry component provides the correct redirection and COW (copy-on-write) operations merging 3 elements into a single registry view:

  • The native registry

  • The immutable staged package registry (built from the registry.dat file within the package)

  • The user and machine COW registry

Like with file assets, the view of each merged registry is done for each package and package group (virtual environment)

Functionality

At runtime, registry operations are hooked, and special sauce is made to ensure the redirection, registry merging, and copy-on-write operations of changes are done.

Registry reads are done in an ordered approach by layer and you can easily confirm this with Process Monitor. The order is:

  • The COW registry is read first

  • Followed by the Package Registry (constructed from REGISTRY.DAT)

  • Finally the native registry for the requested location.

For registry writes, things are much simpler. Registry writes always go to the COW location corresponding to the original key that was opened. Registry data that is written to the user’s roaming registry is tokenized so that it is portable across machine boundaries.

Bear in mind, this is based upon the predication that the registry location is viewed as virtual (opaque) to begin with and was not excluded or configured to be translucent in the sequencer. There are special pointers contained in the registry that will govern opacity that I will mention later.

 

Registry COW Locations

The Virtual Registry will manage and track all COW locations for registry storage. The locations will vary depending on security context and type.

Roaming User Registry COW data will go here:

  • HKCUSoftwareMicrosoftAppVClientPackages<GUID>REGISTRY

  • HKCUSoftwareMicrosoftAppVClientPackageGroups<GUID>REGISTRY

Non Roaming User Registry and non-elevated Machine Registry COW data will go here:

  • HKCUSoftwareClassesAppVClientPackages<GUID>REGISTRY

  • HKCUSoftwareClassesAppVClientPackageGroups<GUID>REGISTRY

For Machine registry data coming from elevated processes, the Registry COW data will go here:

  • HKLMSoftwareMicrosoftAppVClientPackages<GUID>

  • HKLMSoftwareMicrosoftAppVClientPackageGroups<GUID>

 

Special Registry Key and Value Metadata

You may have noticed that for some keys, you will also see some additional data:

This data is not on all keys but when it is, it is reflective of the specific state of the key when responding to registry operations. Particularly, whether the key was previous deleted or if it was configured to be opaque and not merged with the other registry layers.

  • 0x00010000: Key Deleted

  • 0x00020000: Value Deleted

  • 0x00040000: Key Opaque

If the value above also contains a 1 at the end of the type, this means there are sub keys present stored within the value data.

Purging the COW

The Registry COW data is purged along with the rest of the user state when the package has been deleted or when the package is repaired with the –userstate switch.

Categories: Uncategorized Tags: , , ,

App-V 5: Do you Still have to Run Process Monitor within the App-V Bubble when Troubleshooting Applications?


If – by that question you want to know if you must start an instance of Process Monitor within the virtual application like you did in 4.x – no. You could run process Monitor inside the virtual bubble, but it will not yield you much more results. The reason behind this is simple: unlike previous versions of App-V, the REAL registry as well as the native file system – NTFS – is used in App-V 5.

In App-V 4.6 and earlier, if you did not launch process Monitor in the virtual application’s environment (usually through a command prompt) all you would capture related to the virtual application would be operations to file and registry resources outside the virtual environment. In Version 5, running Process Monitor as normal will capture access to the actual locations including registry, package store, as well as VFS (Virtual File System) COW (Copy-on-Write) locations. Why just that? Because that’s where things “actually” are located. What you will have to understand is that once the operations to where the application “thinks” it is located has been hooked in the file system and/or registry every subsequent operation will continue as such to include operations only to the:

  • Package Store

  • Integration junction points to the Package Store

  • Actual package Store paths

  • VFS COW (Copy-on-Write) checks and locations

You can see examples of these in the following screenshots from Process Monitor:

Procmon and File Operations:

As you can see above, the query to the initial location of where the virtual applications thinks it is browsing (C:program Files (x86)Java) is clearly not natively where it thinks it is. The App-V engine picks up for this (through relationships in memory) and the operations are redirected to the appropriate converged locations in both the User VFS COW and Package Store.

Procmon and Registry Operations

You will see similar operations when tracing specific activities to registry entries. First, where the application thinks it is supposed to be located followed by subsequent operations to the actual state-separated locations in the actual registry.

Why does Process Monitor not show every Single Operation to “Virtual Paths”

The answer is quite simple – because the App-V Client takes care of all of this behind the scenes which is why you will need to have access to and understand the FileSystemMetadata.xml file as it contains all of the file system mappings for both non-tokenized paths and tokenized paths. The easiest and most automatic relationships are the KNOWNFOLDERID paths which automatically resolve to App-V tokenized paths in memory. For non-tokenized paths, it is handled differently on process creation.

Altitude Adjustment of ProcMon Driver

When you look at the altitudes of the App-V file system drivers and their relationship to the driver altitude, you can see that the Procmon driver sits at a lower altitude by default.


This might make you explore the possibility of raising the altitude to see if Process Monitor will capture more information. Please be careful doing this as this could create problems and system instability. Altitudes are managed and allocated by Microsoft  (https://msdn.microsoft.com/en-us/library/windows/hardware/dn641617.) When developers want to register altitude locations for their filter driver, they fill out a special request form. That is how tightly controlled they are in order to prevent instability.

 

App-V 5: On Application Launch

September 4, 2014 3 comments

In previous blog posts, I discussed what happens when a package is added/configured, published, and when streamed. But what happens when an application is double-clicked? App-V needs to determine if the application needs to be virtualized, and if so, which virtual environment. The Shortcut is launched. The Junction point is parsed to the right version location in the package cache. The executable’s process is created but there will be a process notify callback by the App-V Virtual Environment Manager (AppVVemgr.sys) to determine if the process should be virtualized or not. Now, I previously blogged about this part of the process last year and gave a convoluted diagram (http://blogs.technet.com/b/gladiatormsft/archive/2013/09/12/app-v-5-to-virtualize-or-not-to-virtualize-how-app-v-5-answers-that-question-when-you-launch-a-shortcut.aspx) which was met with entertaining emails.

Assuming the application is going to be virtualized, after that workflow has completed, then either the AppVEntSubsystems32.dll or the AppVEntSubsystems64.dll is injected into the virtual process depending on the bitness of the application. Of course, it will always be the 32-bit DLL if working exclusively on X86 platforms. You will notice that once an application is running, there are a few ways you can tell if an application is running virtualized or not. I’ve previously mentioned ListDlls.exe from Sysinternals but you can also use Process Explorer, but not for the reason you may think. The “Virtualized” tab within Process Explorer refers to whether the application is being run within WoW64 (for 32-bit applications) rather native x64. Nothing to do with App-V just as the “Package Name” field refers to the base AppX package and applies to modern applications.

But you can use other features of Process Explorer to determine if a process is virtualized with App-V. For example, you verify the path and command line as they will show launched from the location of the package cache.

That is not completely full proof because you could also have an application that is locally installed but has been configured to run inside a package’s virtual environment. This also means the application will have the AppVEntSusbsystemsXX.DLL injected as well. So just search for that. You can do it through the stack itself but that will require you to walk to the individual thread stacks.

Instead you could just simply do a search in Process Explorer for the DLL and you will get a response back of every process for which that DLL has been injected

  

Besides injecting this DLL into the process, several other events are taking place. The App-V Virtual Environment Manager will also create all of the elements needed for that virtual environment including the attaching of the VFS COW Filter driver to the appropriate volumes and send the COW registry and file mappings to the driver assuming they have been staged properly – otherwise, that must also take place. The Virtual subsystem controller must also establish the subsystems with the proper settings based on the packages catalog (COM, Objects, etc.) which is why it is mandatory that these settings match when you are creating and enabling Connection Groups as the settings for the entire Package Group Catalog would also have to be created should this be the first application that is launching in a published Connection Group.

AppV 5: Important Considerations and Ramifications for Package Upgrade and VFS Write Mode

August 30, 2014 3 comments

If you are running any version of the App-V 5 client or Sequencer prior to App-V 5.0 Service Pack 2 Hotfix 4 – stop reading. This does not apply to your environment. If you are running HF4 or sooner, you need to have a good understanding of the net effects of toggling VFS Mode on and/or off during package upgrade.

VFS Write Mode was introduced to (my words) “fix bad brake jobs” when it comes to application development. Applications that need to be able to read and write configurations files in normally protected directories or have modified ACL’s for standard users in order to write to program locations that only administrators would normally have access to (Notepad++ has a way to go back and forth between good and bad program configuration – http://blogs.technet.com/b/gladiatormsft/archive/2014/05/24/app-v-5-on-sequencing-using-tokenized-paths-pvad-s-vfs-and-vfs-write-mode.aspx.)

While VFS Write Mode involves setting a specific attribute within the package manifest, the effects on how the VFS (Virtual File System) and COW (Copy-on-Write) filter are handled for the user are significant. As a result, making any changes to this setting during package upgrade could have some effects.


Scenario 1: Package Upgrade where the VFS Write Mode setting is not changed

This one is pretty straight-forward. No considerations are needed. Nothing is changed in how the VFS handles things.

 

Scenario 2: Package Upgrade where VFS Write Mode setting is turned on when previously not enabled.

When this happens and directories previously existed in the COW but there was no “S” directory, then the original directory will be renamed to the “S” version and the permissions appropriately adjusted. The new “regular” directory will be created. So instead of just one directory (i.e. ProgramFilesX86) there will be two (adding ProgramFilesX86S.) For more information on the “S” directories, please refer to my previous post here (http://blogs.technet.com/b/gladiatormsft/archive/2014/08/29/app-v-5-on-the-now-sometimes-immutable-package-cache-location.aspx.)

 

Scenario 3: Package Upgrade where VFS Write Mode setting is turned off when previous enabled.

In this scenario, there will likely be existing two structures: directories appended with “S” and the regular directory with the relaxed permissions. When this happens, the relaxed permissions directories will be deleted and the “S” directories will be renamed back to their original names. If this is ever done unintentionally, you can imagine some of the issues that may happen so be careful doing this.

 

I would advise always avoiding Scenario 3. Scenario 2 will be quite common as many sequencers and packagers are now upgrading their pre-SP2 HF4 App-V packages in order to take advantage of the new VFS Write mode feature. The question I am getting asked a lot lately is whether it is better to re-sequence with VFS Write Mode on or to simply just perform a package upgrade. I would advise trying the package upgrade first to turn on the value. In most cases, this should work – but as always, I await your comments.

App-V 5: On the (now sometimes) Immutable Package Cache Location

August 29, 2014 6 comments

When you add/configure and publish a package in App-V, the primary file assets and source registry hives are stored in the package cache location. It defaults to %PROGRAMDATA%App-V. The package cache will store packages in a <PACKAGE_GUID><VERSION_GUID> directory structure for each package added to the machine regardless of targeting. This package cache will exist for the purposes of marking key assets in both stream-to-disk scenarios in stream-to-memory (Shared Content Store) scenarios.

In many cases you may want to redirect the location of the cache without relocating the location of %PROGRAMDATA%. This can be done by changing the location of PackageInstallationRoot through the registry, PowerShell, or through Group Policy. Technically you might be able to get away with not having to restart the client if you do it through PowerShell, but I would advise that you do it anyway.

Registry Location:

  • HKEY_LOCAL_MACHINESOFTWAREMicrosoftAppVClientStreaming
  • Value: PackageInstallationRoot
  • Data Type: REG_SZ

PowerShell:

Set-appvclientconfiguration –PackageInstallationRoot <Path>

You may have noticed that with the defaults, nothing shows up (even the default directory) until the first package is created. Once the first package is configured, the directory is created as a hidden directory with administrators, system, and trusted installer having control with the directory ownership going to SYSTEM. Because of the ACL’s, removing this out of band requires taking ownership and resetting permissions prior to deletion if you ever have to manually remove this.
If you change the location to another directory or another drive without pre-creating and setting the proper permissions, you will notice that you encounter add/publish package failures with error 0c-80070005. At the bare minimum, you would need to redirect to a directory where system does have full control. If the package cache is ever moved without being properly reconfigured, the App-V client service will fail to start.

If you redirect the App-V package cache to a persistent disk in non-persistent scenarios and also redirect catalog data, you can create situations where stale catalog data is trying to be sourced upon reversion. Non-persistent scenarios must avoid situations where catalogs from previous installations pre-exist in the redirected package cache locations. If you redirect the entire %ProgramData% directory to a persistent disk, you will likely run into this in non-persistent (or in the case of the use of a persistent disk.)

If you are suspecting that you are getting this issue, you can confirm it by enabling advanced debug logging of the Client-Orchestration, Client-Service, and the Streaming Manager logs. The Client-Orchestration and Client-Service log will throw error 0XA560212C-0x3. The StreamingManager log will give the following error:

Get DACL from directory failed. Error: The system cannot find the path specified. <PATH>

The quickest solution is to remove the old catalog from %ProgramData%MicrosoftAppVClientCatalog and restart the App-V Client service.

How does VFS Write Mode fit in?

With the redevelopment of the virtual file system including a new user-mode VFS and the COW (copy-on-write) filter, new challenges arose and this package cache format preventing applications from functioning correctly in certain scenarios such as:

  • Needing to write to Common App Data or a virtualized %PROGRAMDATA%
  • Creating Folder and files beneath %ProgramFiles%
  • Adjusting permissions on User COW locations (which, in user targeting, mirror the native directories.)

Enter VFS Write Mode. This reverts back to the notion that the user who owns the COW (which for global would mean potentially any user) gets full access to the top-level COW directories it would normally have access to. Additional compliance in line with Windows Resource Protection and UAC do remain in effect however. To differentiate, there will be directories denoted with an “S”. Standard users will read and write from the regular directory (with relaxed ACL’s) while elevated processes will read and write to the “S” version.


 
The VFS attribute is a very simple one that is stored in the manifest and can also be used with sequencing templates. It is not yet available with dynamic configuration. Yeah . . . that might/ought to get changed.

But what happens when you complicate things with Connection Groups?

The expected behavior for Connection Groups is actually pretty simple. If you add a package with VFS Write Mode turned on to a Connection Group, then the net effect of that will be the entire Connection Group has this turned on since the Connection Group shares a common COW. This setting will persist.

COW Restrictions

COW restrictions by extension have not changed. All of the executable extensions and common application binary extensions are still restricted from being modified even with VFS Write Mode turned on. These include: ASP, BAT, CER, CHM, CMD, COM, CPL, CRT, DLL, DRV, EXE, FON, GRP, HLP, HTA, JS, LNK, MSI, MSP, MST, NLS, OCX, PIF, REG, SCR, SYS, URL, VB, VBE, VBS, WSF, and WSH

Categories: Uncategorized Tags: , , , , , ,

App-V 5 SP2 Application Publishing and Client Interaction – Now Available!

January 19, 2014 Leave a comment

 A much desired, long-awaited, and highly anticipated white paper was released last week. If you are looking to understand how virtual applications are added, published, and delivered from publishing servers (especially the differences from previous versions) look no further. The document is available here!

http://www.microsoft.com/en-us/download/details.aspx?id=41635

Categories: Uncategorized Tags: , , , , ,

App-V 5: On Explaining this New VFS to Softgrid Users

November 4, 2013 5 comments

There is an expression that is based upon a well-known book – “Who moved my cheese?” Ah, change – it can be hard. I will confess, there was a lot of “cheese moving” in the redevelopment of App-V 5 from my perception. I’ve
spoken about some areas where changes have been significant. I find myself doing a lot of explanation to Softgrid users. When I speak of Softgrid users, I speak of those who were brought up in what I now call the “legacy SFT world.” This was the world of the SFT file; the world of the SFTMIME and SFTTRAY commands; the troubleshooting of the SFT listener service and the SFTLOG file. This went all the way to App-V 4.6. It is still in use, still supported, and it will be here for a while. But as of App-V 5.0, that era is now starting down its road into the sunset. If you are looking to survey App-V for the first time, you should be looking into a modern solution: App-V 5 and beyond.

To continue on a point I alluded to earlier regarding change, I wrote an article on scripting several months back. Scripting was indeed an area where there was tremendous change. I felt it was needed as I was hearing numerous stories about “how scripting just doesn’t work in App-V 5.” What bothered me the most was that I was hearing this from experienced users of App-V 4. I laid down as best I could a guide for transitioning scripts from 4.x to 5.x. Not only were the mechanics different, but a lot of concepts had changed, because we now had more options. It was change. Change for the better.

With a new VFS, there are also changes, but the concepts are very similar. It is the implementation that is drastically different. I’ve noticed something interesting. With this new VFS, there is a huge desire to understand what has changed and what has not changed. There is also the assumption that since the way App-V handles file assets has been completely re-written, then all of the fundamental basics should be thrown out the window. Sometimes, one can get lost when there as an attempt to over complicate something. And believe me, I’ve read some over-complicated interpretations out there in the blogosphere. Fundamentally, at a high level there are no major differences. AppV uses a VFS to separate assets from their traditional locations as to not conflict with each other. The concept is not different from 4.x.

Isolation vs. Immutability and State Separation

Isolation implemented differently is still isolation and protection. Moving from a proprietary file system to one that uses native NTFS means that the way AppV “isolates” has to be implemented differently. If you ever look at the Server App-V product, you can actually see a mid-point in this evolutionary process where there was indeed, still a “Q drive” – yet the Q: drive yielded a human-readable file system. With App-V 5, there is a moving of these “Q:
drive assets” or %SFT_MNT% assets into an overlay of file assets that are state separated where the base package remains immutable or read-only – just as the assets within %SFT_MNT% which were cached into an actual file called SFTFS.FSD in 4.x.

Advice for the Softgrid Users:  Learn by aligning the concepts

Transitioning is easier when you can map where the concepts carry over. Then all that is needed is to focus on the details of transitioning how you would virtualize your applications in v5, especially those that required customization and advanced sequencing techniques.

SFT_MNTPackageRoot vs. AppVPackageDriveAppVPackageRoot

The package root in v4 (which we referred to as SFT_MNT due to the variable %SFT_MNT% reflecting the package root of an application) was laid down in the virtual environment as Q:RootDir. Applications outside the virtual environment could not see this without special configuration. All cached assets went here in their own separate folders which would all actually be stored inside a read-only file system file called sftfs.fsd (by read-only, I mean immutable – the changes and modified files go somewhere else.) In V5, it is essentially the same. They are just laid down in human-readable file system assets. It is just not isolated. However, it is still read-only. It is still
immutable. This is why you cannot change anything – by anything we also mean permissions. When you add a package through PowerShell, or a package comes down on the ConfigurePackage function (using the publishing server) regardless of the target publishing, a “Package Root” is laid down with a series of shadow files and directories.

This will be in the PackageInstallationRoot configuration item which defaults to %PROGRAMDATA%AppV but it can be redirected if needed by changing PackageInstallationRoot in HKLMSoftwareMicrosoftAppVClientStreaming in the registry. Directories and markers will be laid down as well. These markers are actually shadow files. Do not confuse these with sparse files. Sparse files will be used if you are implementing the Shared Content Store. When you stream the package, the shadow files will become the streamed-to-disk assets. They will be immutable, or read-only. The integration components, including links and shortcuts, will not be laid down until the publish event has occurred. The integration link placement will depend on how the actual package has been targeted for publishing.

Targeted Publishing – A Difference

It is with publishing that things get different. The integration components (what serves as the target for linking the OS environment and ultimately the user) for this package will be placed in IntegrationRootUser if it is published to the user and IntegrationRootGlobal if it is published to the machine (globally.) These locations can also be controlled by setting those values in HKLMSoftwareMicrosoftAppVClientIntegration.

For those packages that are targeted to the user you will see the symbolic links to the package GUIDs under IntegrationRootUser which will reflect the beginnings of the user state data, the user mode VFS, and the elements that are allowed through the COW (copy-on-write-filter.) In essence, the data that would have normally be encapsulated inside of the PKG files in Softgrid. With user publishing, you will see here exclusively the publishing
integration elements which would normally be under %PROGRAMDATA%MicrosoftAppV when they are published globally. Even with global publishing, the users will maintain settings for their user state within %LOCALAPPDATA%MicrosoftAppV or IntegrationRootUser.

 

How AppV 5 Manages File-based user data vs. PKG’s

We used to call these PKG files. We use to have both user state data and global package data inside these PKG files. This data is managed differently in that they are no longer isolated into PKG files. AppV lays down these assets into human-readable files but everything is still state separated. But for the application, the virtual view is no different conceptually. The virtual view is the combination of namespaces in V5 just as it was in V4 (where the
virtual view was a combination of read-only immutable assets (within the FSD) and read-write global and user assets (within the PKG.)

Caching is now mounting

Which means, to pre-stream, or pre-mount, you simply run a “Mount-AppVClientpackage” cmdlet. Otherwise, the operation is similar in the mounting occurs on demand when you stream the application on first launch. Slowly, as the files are fully downloaded, the shadow files become fully cached files. The icons switch back to a normal icon in the PackageInstallationRoot. THAT IS – if you are using the traditional “stream-to-disk” process. If you are
implementing the Shared Content Store or “stream-to-memory” you will not be caching 85-90% of the file system assets. Those sparse files remain as such in the PackageInstallationRoot. NOTE: If you run the Mount-AppVClientPackage even in shared content store mode, it will fully cache those files to disk.

Now Why Can’t I set ACL’s on the Package Files beneath PackageInstallationRoot?

Applying permissions to the files and directories beneath PackageInstallationRoot is not possible because, as I mentioned, earlier, these files are read-only and that also includes the extended attributes. You can however set permissions on folders and files located in the user mode VFS located beneath IntegrationRootUser (which itself is beneath %LOCALAPPDATA%.) Pre-staging these permissions does require some work. Some in the community
have come up with great ideas. I like the script by Dan Gough in particular (located here: http://packageology.com/2013/06/file-permissions-app-v-5/) The reason why it has to be done this way is due to the new way in which file
assets are implemented using native NTFS instead of the previous methods.

Conclusion

Change can be hard. But not so much if all you are having to do is learn new technique. Conceptually, not a lot has changed but because Windows as a whole is evolving, applications and application formats are modernizing,
it is important that App-V evolves as well as a key tool for that modernization. This means that certain things may change when it comes to minor elements such as permissions. In a later post, I will dive further into the intricacies of how files are handled and how this understanding can help us troubleshoot when needed.

I sense the comments will come in 3 . . . 2 . . . 1 . . .

Categories: Uncategorized Tags: , , , , , ,

App-V 5: On that issue with Firefox where you cannot save preferences

October 24, 2013 6 comments

If you have had a chance to sequence Firefox within App-V 5, you probably have encountered an issue where none of your preferences seem to be retained after exiting Firefox. Anything from loaded plug-ins, to tab preferences, to even the home page seem to always revert back to the default. This scenario is pretty easy to reproduce with a clean sequencer template and the default App-V sequencer exclusions. I used Firefox 18.0.2 installer to demonstrate this issue.

What is happening?

A user launches Firefox and proceeds to set all of their customizations (Home Page, Tab settings, etc.) with in the options menu (using the Firefox menu on the top left corner.)

 

After making these changes, exit out of Firefox (ensure the actual process terminates) and re-launch the application. All of those changes within the options menu are gone and you have been reverted back to the “first-launch” Firefox experience. A repeat process and it happens again. And again.

Troubleshooting what is actually happening

So for me, capturing two separate process monitor traces usually helps me sort this out. I started a Process Monitor trace to capture launching Firefox and saving a simple preference. Then I stopped the capture, saved it, and re-launched process monitor in order to capture the exit and re-launch for Firefox. I do this a lot and I will work from the middle. My initial guess was that the issue with saving the preferences actually happened on exit. Before I looked at the process monitor trace, I needed to know how these preferences are actually being stored (in the registry, within a file, etc.)

I found that the preferences are being saved in a file called prefs.js. Interesting. So I pulled up the first process monitor PML I captured and filtered for a path that contained prefs.js.

 

I found numerous CreateFile, QueryBasicInformation, and CloseFile operations on prefs.js as I was walking the trace through the normal VFS hook/walk functions (Integration symbolic link, “gold” package store, user-mode VFS, etc.) The CreateFile operations seemed to be normal when the desired access was READ, READ EA, SYNCHRONIZE and are successful. However, as I moved on down at the time I started changing settings and clicking OK, I saw that a CreateFile to the same Prefs.js file was failing with ACCESS DENIED. This time the desired access also included a GENERIC WRITE. Odd, these should normally be supported too.

 

Looking at the second trace where I captured closing and reopening Firefox, I found numerous instances of the failed writes to the prefs.js file. This prefs.js file was located beneath the user’s roaming %APPDATA% folder.

 

 

 

So again, what exactly is happening and why?

This requires a little bit of background. As you may have discovered, virtual assets in App-V (files, folders, registry keys, etc.) are not made invisible. They are not isolated in the sense they are in human readable formats – yet they are still state separated. When processes are hooked by App-V, they still see things virtually as they would in previous versions of App-V. It’s just the job of App-V behind the scenes to redirect everything to the appropriate location.

When you look at the file layouts of an AppV Package from within native explorer, you will see a read-only package beneath the %PROGRAMDATA%AppV folder. If you look inside C:Program Files, you will not see the Mozilla Firefox subdirectory. However, if you configure Explorer to launch inside an App-V bubble, you can view the VFS directly as the application sees it. All you have to do is first configure explorer to launch inside a separate process. In any explorer folder window pull up “Folder and Search Options,” select the View tab, and check the box to “Launch Folder Windows in a Separate Process.”

 

Then using the /APPVVE or /APPVPID extension launch a command prompt inside the virtual environment you would like to view. Then you can spawn an explorer process inside the App-V bubble.

 

In the figure below, the same folder is viewed with the top window coming from a native explorer process and the bottom from an explorer process running inside the virtual environment of Firefox.

 

Now using the same bottom window running in the virtual environment, I navigate over to that particular directory where the prefs.js file is.

 

 

 I open and try to modify it. It prompts me to save it into a different location. The issue reproduces in this manner as well.

 

While running this process, we are viewing the true virtual package view in which we are combining and merging the view of two different namespaces. The base package or “GOLD Package” and the “User Store” (within the User Profile in %LOCALAPPDATA%.)

While this demonstration was long, I did it to demonstrate something implemented in the complex App-V 5 VFS is a component known as the VFS COW (Copy-on-Write) filter. The COW manages per-user data and package wide data that are stored per-user (in the case of applications that are targeted to the user.) Writes are redirected from the base package location to a COW location inside the user’s profile. This makes it possible for users to have different views of the same package and all changes are isolated between users. It also prevents users from tampering with App-V packages from outside the virtual environment yet it will STILL allow Anti-Virus applications to scan and REMOVE viruses from a virtual application should the rare case in which one would arrive via an App-V package. The mappings maintained inside the registry as shown here for FireFox (based on its GUID.)

 

One particular item that is also a feature of the COW are the COW Exclusions. Copy-on-Write functionality will not occur for certain file types. In the case of Firefox, it is not a problem with it writing to a COW location, but the file type in which it is trying to write.  Files with extensions of EXE, CMD, MSI, COM, BAT, JS, VBS, DLL, and others cannot be modified because it will trigger a new instance of these files within the user’s profile and these extensions are normally associated with executable code or scripts.

How to resolve?

You have a few options here. You can simply exclude the folder containing the file from sequencing or you could take advantage of user state management solutions that work with App-V (UE-V, RES, AppSense, FlexProfiles, etc.)

Is this a Bug?

In the context of App-V, no. Is this a victim of the design of App-V 5? Before you rush to judgment, let me offer you this scenario and that might shed light on who you should be directing your potential outrage against. This particular issue boils down to one simple thing – Firefox saving preferences to a JS file. It is never a good idea for enterprises to have applications that are executable (or executable scripts) to reside in the context of the user’s profile. This is why most enterprises resent Google Chrome and other applications that install executable code into user profile locations. Historically, the AppData folders were also targets of AdWare and Spyware.

Categories: Uncategorized Tags: , , , , ,