Archive

Posts Tagged ‘sequencing’

App-V: How the Sequencer Detects an Existing Package was Created

October 26, 2014 Leave a comment

One of the most consistent recommended practices throughout the history of App-V sequencing is to always sequence on a clean sequencer. Starting with App-V 4.6 SP1, a detector was put in to warn users if they were about to sequence on a machine where a previous package was sequenced.

 

When the App-V sequencer is installed, it will create a registry key called ReadyBit located beneath HKLMSoftwareMicrosoftAppVSequencer and set this value to 1. As soon as a sequence begins to go forward (creating a PVAD, Package Name, etc.) this bit will be set to 0 to inform the sequencer that we are no longer working with a clean sequencer.

So, the obvious next question would be, should we be manually resetting this bit. Only in certain circumstances. Let’s say you gave an incorrect package name or specified the wrong PVAD and went forward with the wizard and accidentally began monitoring. You had yet to install any binaries but you realized you made a mistake. Is there another option other than reverting to a previous snapshot of the virtual machine? What if you were doing this on a physical workstation? Yes, you can exit out of the sequencer, set that registry setting manually, and restart the sequencer and not have to worry about having that warning pop up during the computer preparation stage.

Please note that this is the ONLY time I would recommend doing this.

App-V 5: On Environment Variables

October 26, 2014 8 comments

Environment variables are managed on a per-package basis in App-V 5 just as they have with previous versions. During sequencing, the Environment Variable Virtualization Subsystem (say that 10 ten times fast) will capture snapshots of the user and system variable list before and after the monitoring process. Then lists are compared and any differences are added to the package manifest. This means the package will default to the variable data entered in during sequencing. The variables are then stored in the package manifest using the following format:

<appv:Include>

<appv:Variable Name="TMP" Value="[{LocalAppData}]Temp" />

</appv:Include>

Where the “Name” is the variable and “Value” is the new, tokenized (if necessary) value of the environment variable.  On the client, the named variable will be set to the new value within the virtual environment and properly translated.

The one time there is an exception to this process is when the PATH value is being handled. Applications never “set” the PATH variable – they only append, or add to it therefore if changes to the PATH value are detected during sequencing, the changes will be captured and will appear slightly different in the manifest, with a self-referential value. The PATH variable that is captured in manifest will look similar to:

<appv:Variable Name="PATH" Value="<dirpath1;%PATH%;dirpath2" />

Where dirpath1 would be a prefixed Path segment to the current PATH and dirpath2 is a suffixed path segment to the current PATH.

The sequencer can also detect the deletion of existing variables and will apply those in the manifest using the following format:

<appv:Delete>

<appv:Variable Name="TEMP" />

</appv:Delete>

In this case, the Environment Variable TEMP will be deleted within the virtual environment of the application.

These captured values can be modified and overridden using dynamic configuration. An administrator can modify DeploymentConfig and/or UserConfig to edit Environment Variables. The <Include> element beneath the <EnvironmentVariables> section is where new or modified variables should be placed. The <Delete> element is where values need to be removed. In the example below, the PATH variable is modified, two additional variables are added and one will be deleted (if exists.)

 

The Virtual Environment Variable Subsystem reads the package manifest and the option deployment and user dynamic configuration files, and produces two lists – variables to add and variables to remove.  This is all done when the application launches. As the catalog documents are processed, newly processed values overwrite previously processed values, so last writer wins. It is important to make sure all needed variables from the manifest are transferred over to the dynamic configuration files when leveraging them to change variables post-sequencing.

As with sequencing, there is an exception and it again involves the PATH variable. The PATH variable is injected after a process has completely loaded. This is why it is not a good idea to try to use the PATH value to help a virtual application to find a static binary dependency (even inside the immutable package cache.)

Caveat with Connection Groups

An area that confuses sequencers is how environment variables are handled with Connection Groups. Environment variable lists overwrite each when combining packages into Connection Groups. Like with registry convergence, variable convergence is a last writer win scenario in the case of overlapping variable names. The package with the highest order within the connection group descriptor document wins variable name conflicts.

Exception with Multi-String Values

Be advised though that some variables are treated specially, as they have been identified as having multi-string values.  If a multi-string variable is present in multiple packages, the resulting value is the combination of values from all packages, instead of the last writer winning. The list of variable names which are treated as multi-strings is stored in the registry at HKLMSOFTWAREMicrosoftAppVSubsystemEnvVarMultiStrings

Exception with Cross-Bitness

App-V deploys special additional tokens used to denote information that has to be adjusted depending on the bitness of the sequencing machine and the correlation of the client machine. These tokens apply to environment variables impacted by Wow64 redirection. They enable the proper translation when an application that was sequenced on a 32-bit machine gets deployed on a 64-bit client machine.

The two are:

  • AppVEnvironmentVariableCommonProgramFiles: Can be %commonprogramfiles% or %CommonProgramFiles(x86)%

  • AppVEnvironmentVariableProgramFiles: Can be %ProgramFiles% or %ProgramFiles(x86)%

The rules for this are as follows:

  • If the application is a 32-bit application, sequenced on a 32-bit operating system, deployed on a 32-bit operating system, then these tokens are expanded back to the original values.

  • If the application is a 32-bit application, sequenced on 32-bit operating system, deployed on 64-bit operating system, then these tokens are expanded back to X86 versions of the values.

  • If the application is a 64-bit application, sequenced on 64-bit operating system, deployed on 64-bit operating system, then these tokens are expanded back to the original values.

Verification of Virtual Environment Variables

When testing environment variables within a virtual application package, the environment variables will not be visible outside of the virtual environment. That is why you will see no changes when you list the variables using the internal “set” command:

 

However, when you open the command prompt inside of the virtual environment, you will see the variables from above:

 

 

 

 

AppV 5: Important Considerations and Ramifications for Package Upgrade and VFS Write Mode

August 30, 2014 3 comments

If you are running any version of the App-V 5 client or Sequencer prior to App-V 5.0 Service Pack 2 Hotfix 4 – stop reading. This does not apply to your environment. If you are running HF4 or sooner, you need to have a good understanding of the net effects of toggling VFS Mode on and/or off during package upgrade.

VFS Write Mode was introduced to (my words) “fix bad brake jobs” when it comes to application development. Applications that need to be able to read and write configurations files in normally protected directories or have modified ACL’s for standard users in order to write to program locations that only administrators would normally have access to (Notepad++ has a way to go back and forth between good and bad program configuration – http://blogs.technet.com/b/gladiatormsft/archive/2014/05/24/app-v-5-on-sequencing-using-tokenized-paths-pvad-s-vfs-and-vfs-write-mode.aspx.)

While VFS Write Mode involves setting a specific attribute within the package manifest, the effects on how the VFS (Virtual File System) and COW (Copy-on-Write) filter are handled for the user are significant. As a result, making any changes to this setting during package upgrade could have some effects.


Scenario 1: Package Upgrade where the VFS Write Mode setting is not changed

This one is pretty straight-forward. No considerations are needed. Nothing is changed in how the VFS handles things.

 

Scenario 2: Package Upgrade where VFS Write Mode setting is turned on when previously not enabled.

When this happens and directories previously existed in the COW but there was no “S” directory, then the original directory will be renamed to the “S” version and the permissions appropriately adjusted. The new “regular” directory will be created. So instead of just one directory (i.e. ProgramFilesX86) there will be two (adding ProgramFilesX86S.) For more information on the “S” directories, please refer to my previous post here (http://blogs.technet.com/b/gladiatormsft/archive/2014/08/29/app-v-5-on-the-now-sometimes-immutable-package-cache-location.aspx.)

 

Scenario 3: Package Upgrade where VFS Write Mode setting is turned off when previous enabled.

In this scenario, there will likely be existing two structures: directories appended with “S” and the regular directory with the relaxed permissions. When this happens, the relaxed permissions directories will be deleted and the “S” directories will be renamed back to their original names. If this is ever done unintentionally, you can imagine some of the issues that may happen so be careful doing this.

 

I would advise always avoiding Scenario 3. Scenario 2 will be quite common as many sequencers and packagers are now upgrading their pre-SP2 HF4 App-V packages in order to take advantage of the new VFS Write mode feature. The question I am getting asked a lot lately is whether it is better to re-sequence with VFS Write Mode on or to simply just perform a package upgrade. I would advise trying the package upgrade first to turn on the value. In most cases, this should work – but as always, I await your comments.

App-V 5: On Sequencing Using Tokenized Paths, PVAD’s, VFS, and VFS Write Mode

May 23, 2014 7 comments

Update 12/5/2014: The PVAD is now considered optional due to changes in App-V 5Service Pack 3. Read more here: http://technet.microsoft.com/en-us/library/dn858700.aspx#BKMK_pvad_hidden

 

When sequencing applications, you often never know how the application was written with regards to what paths are actually referenced via hard-coded paths or through tokenized, or well-known paths or KNOWNFOLDERIDS. In previous versions of App-V and Softgrid, these paths were referenced using %CSIDL% variables. These are well-documented on MSDN (http://msdn.microsoft.com/en-us/library/windows/desktop/bb762494(v=vs.85).aspx). If the application was developed to reference CSIDLs, they usually function when virtualized just fine with App-V. CSIDLs are still supported for backward compatibility, but going forward starting with App-V 5, a different tokenization process is used to align better with KNOWNFOLDERIDs. The new App-V Tokens and their default locations upon installations are well documented in the App-V 5 sequencing guide:

http://download.microsoft.com/download/F/7/8/F784A197-73BE-48FF-83DA-4102C05A6D44/App-V%205.0%20Sequencing%20Guide.docx

Tokenization helps to remove a specific application’s tie-in to specific folder paths and to allow for greater portability especially when these KNOWNFOLDERIDS have been redirected for specific configurations.

It is important to understand that for sequencing applications that will be used for Connection Groups or to better take advantage of the App-V 5.0 SP2 HF4 VFS Write Mode, all files will need to be located in the VFS structures. This will drastically affect the sequencing process.

Sequencing to a PVAD, Installing to the PVAD

If you specify a Primary Virtual Application Directory and then install to that same location, the base directory of installation files will not be tokenized. For many older applications, this may be the best route to take. For example, if the Primary Virtual Application Directory (PVAD) is specified as C:DIRECTORY1 and the sequencer monitors an install during sequencing to C:DIRECTORY1 then everything created beneath DIRECTORY1 goes beneath Root and will not be tokenized or located beneath the VFS folder. There may be additional files that do in fact get written to tokenized paths and they will be tokenized but you will now have a mixture of both “rooted” file assets and VFS file assets.

In the example below, Notepad ++ is sequenced to C:Notepad and installed to C:Notepad:

As you can see above, the majority of the files appear below the Root structure and are not contained with the VFS tokenized directories. Also notice the folders beneath Root cannot be adjusted for opacity:

  

When the files are deployed to the App-V immutable package cache, they will be laid out the same way.

  

In addition, in order to troubleshoot and navigate the file system within the bubble, non-VFS folders do not appear in directory listings.

  

You will be able to change to virtual directory by using cd Notepad+

 

Sequencing to the PVAD, Installing to Program Files or a Different Location other than the PVAD (Fake PVAD)

If you specify a PVAD and then install to a different location, all of the files will be written to the VFS location inside the package. For example, if the PVAD is C:DIRECTORY1 and the sequencer monitors an install to C:Program FilesDIRECTORY1 during sequencing, then everything gets put beneath RootVFS and root installation files will get tokenized if they are written to write to a KNOWNFOLDERID or CSIDL during installation.

In the example below, Notepad++ is sequenced using a fake PVAD and installed into C:Program Files

As you can see, all of the files are stored beneath the VFS directory structure.

When files are VFS’ed, you can see the directory listings as normal directories converged with the local directories when you launch a command inside the App-V bubble.

  

What About Using a KNOWNFOLDERID as the PVAD?

This question comes up quite a bit. Many packagers like to match PVAD with the installation directory (especially now that the Q: requirement is gone.) This is fine but understand that when you install to a PVAD during sequencing, all installation files beneath that folder that are not otherwise set to install to another explicit KNOWNFOLDERID or CSIDL path will be treated as rooted files.

For example, if you choose C:Program FilesDIRECTORY as the PVAD and the sequencer monitors an install to C:Program FilesDIRECTORY1 during sequencing, then most files will still be beneath root and NOT VFS except for tokenization of explicit KNOWNFOLDERID paths.


In the example below, when sequencing Notepad++, a PVAD of C:Program FilesNotepad++ is specified.

During the monitoring of the installation, notice the specified install directory is also C:Program FilesNotepad++.

Upon completion of sequencing, you will then notice that the package files are laid out just like any other PVAD-sequenced package.

The base installation files are located beneath Root.

Like the example earlier, when you open up a command prompt inside the bubble, you will not see the root directory in a directory listing.

However, you can still change to it manually inside the bubble.

So What about this VFS Write Mode?

For a lot of applications, especially older applications, they are written to use configuration files beneath the root of the installation directory. Natively installed applications often have to use shims to redirect these modifiable configuration files to safer directories. Combine this with the fact that most packagers are not always sure where to place the installation files in relationship to the PVAD (we cannot always read the minds of developers) and you can run into problems when virtualizing applications. While using shims is possible with virtualized applications, this can drastically increase the sequencing and remediation time for an application.

Enter VFS Write Mode. The App-V team added this “silver bullet” of a feature into HF4 of App-V 5 SP2. Using the advanced configuration options in sequencing, you can now select this option to enable an application to normally open files for modification within the package.

To demonstrate how this can be helpful, once again we will use Notepad++. When you install Notepad++, you have the option to load/write the configuration files from the install directory instead of using the %APPDATA% directory which would be properly tokenized.

Prior to the invention of VFS Write Mode, if you selected “Don’t use %APPDATA%” during sequencing, this is what would happen when you tried to run the program virtualized. First you would get an error message like this:

This would be followed by a crash.

When you troubleshoot the application with Process Monitor, you will notice a series of failures when trying to load the files:

In HF4, you can open up the package in the new sequencer, navigate to the “Advanced” tab, and check the box enabling VFS Write Mode.

As a result, Notepad++ will open and function as expected.

You will also notice in Process Monitor, the file operation to open the XML configuration files in the installation directory (with Read/Write/Delete) is successful.

 

The great thing about VFS Write Mode is that it can be enabled simply by opening up the package in the App-V 5 SP2 HF4 sequencer. You can now put this into your arsenal of App-V AppCompat secret weapons.

Update 12/5/2014: The PVAD is now considered optional due to changes in App-V 5 Service Pack 3. Read more here: http://technet.microsoft.com/en-us/library/dn858700.aspx#BKMK_pvad_hidden

App-V 5: Application Troubleshooting: On the Origins of an Error Message

May 12, 2014 2 comments

Have you ever been testing a virtualized application and it fails with a bizarre application error that is either extremely vague (unknown error) or coupled with some random hex code? Your first reaction usually is “Where’d that error come from?” That is a good reaction to have as it is the first major hurdle in determining how to troubleshoot what has gone wrong. If you have not narrowed down the target of where the application error is coming from, you leave yourself open for spending a tremendous amount of wasted time down “troubleshooting rabbit-holes.”

Is it an App-V Operational Issue?

The first thing you will want to narrow down is whether or not it is an operational issue coming from the App-V Client itself, or is it related to the package (Application not functioning as expected.) I always recommend to first look at the source of the error window. If the error message is originating from a “Microsoft Application Virtualization” window, it is likely an operational issue tied to one of the client engine components or perhaps a streaming issue.


If it looks to be an operational issue, I would advise you leverage some scripts and tools written by Dave Falkus, one of my colleagues in the UK: http://blogs.technet.com/b/virtualworld/archive/2014/04/12/app-v-5-0-etw-tracing-automation.aspx

In addition, one of my previous articles will assist you in dissecting App-V error codes: http://blogs.technet.com/b/gladiatormsft/archive/2013/11/13/app-v-on-operational-troubleshooting-of-the-v5-client.aspx

So I’ve determined it is an error that occurs within the virtualized application itself, now what?

Whether you are isolating an issue that may be caused by virtualization or by bad sequencing, if you have determined the error is coming from the virtual application then it is time to start doing a little reverse engineering with a couple of Sysinternals tools. The first thing I do for an application completely new to me is map out the EXE and DLL launch tree. You can do this by capturing the issue with Process Monitor ensuring that you start capturing at application launch.

You can then run a quick capture filter (I usually load a saved filter with these filter settings)

My filters contain these operation events:

  • Process Create

  • Process Start

  • Load Image

  • Process Exit

If the application is devirtualized on the sequencer, you will be looking to start from the shell process that initiated the Process Create operation (CMD.EXE or EXPLORER.EXE likely.) On a side note, svchost.exe will spawn modern apps, but that is out of scope here. J

If the application is running virtualized on the client, you will see that the AppVClient.exe process interjects spawning the mavinject32.exe or mavinject64.exe process depending on application bitness – followed by the shell creating the process. The example below using virtualized SongSmith displays this in Process Monitor:

Incidentally, if you want to know more about how the AppVClient.exe determines when to run the MAVInjector, read this previous article:

http://blogs.technet.com/b/gladiatormsft/archive/2013/09/12/app-v-5-to-virtualize-or-not-to-virtualize-how-app-v-5-answers-that-question-when-you-launch-a-shortcut.aspx

OK, I have identified the EXEs and DLLs, Now What?

You can do one of two things. You can search the Processes for strings using Process Explorer, or in the case of DLLs and EXE’s, you can simply leverage the Sysinternals STRINGS.EXE tool. The Process Explorer tool is great for finding error strings (by going to the properties of a process, selecting the “Strings” dialog box, clicking the “Find” button.)

The reason I do not use this is because it requires the process to still be active and it is not easy for searching DLL modules if they are not still in memory. I use instead the STRINGS.EXE utility (http://technet.microsoft.com/en-us/sysinternals/bb897439) as it allows for more search flexibility.

Case in point, this bizarre error in Notepad++:

I can confirm the error came from the EXE NOTEPAD++.EXE by using the following command:

Strings-o “C:Program FilesNotepad++Notepad++.exe” | findstr /I scintilla

  

Putting it All Together

When you get that unknown application error, you can then proceed to investigate towards a resolution using a clear path by:

  • Determining whether it *IS* an application error and not an operational error.
  • Determining if it is a sequencing or virtualization issue.
  • Collecting the Process/DLL load/execution order.
  • Identify which EXE or DLL is contains the error string.

 You can then proceed to continue troubleshooting by focusing around that EXE or DLL with Process Monitor, SpyStudio, APIMon, or whatever tool you prefer to further debug the issue.


App-V 5: On Using Sequencing Templates

March 18, 2014 5 comments

Sequencing Templates (.APPVT) files are designed for automating the sequencing of applications. While you can take advantage of some of the benefits of templates with manual, interactive sequencing, be careful making assumptions when sequencing following the importing of a template in the Sequencer GUI. Sequencing Templates are also essential for the upgrading of packages.

Remember this from the App-V Sequencing Guide:

“Templates are also very important for upgrade scenarios.  The Sequencer does not save state so when a new Sequencer session is open and a package is opened for upgrade, the settings are in the default state.  If certain sequencer settings were changed when sequencing a package, the changes will not remain at time of upgrade.  Therefore, it is recommended to save a template for any package that has Sequencer customizations, and re-apply them on upgrade.  A template may also contain additional options such as Package Deployment Settings and Advanced Monitoring Options.”

Creating a Sequencing Template

Creating a sequencing template is pretty straight forward. You launch the App-V Sequencer and first set your advanced options for sequencing. You do this by going to the “Tools” menu and selecting “Options.”

All of the General Items and Exclusion Items can be adjusted using this dialog box. All of these settings will be saved into the template.

If you plan on only using these settings in your template, you can proceed to save as template using the “File” menu to “Save as Template.” However, if you want to include additional settings (for automated sequencing with PowerShell) instead of saving as template, proceed and go through the process of creating a blank dummy package. Make sure you click through to the advanced options so you can configure:

  • Operating System Options
  • Advanced Interaction

 

 

Once you have all of these settings the way you want them then you can proceed to save the template. Notice you will get a specific alert when doing so.

 

While it implies that the additional settings (OS, COM, objects) will not be saved in the template, you will find that they are, in fact, saved. What the effect of this message is any settings other than General Options or Exclusion Items will NOT be imported if you import the template into the sequencer GUI for the sequencing of a new package.

 

 

All of the settings will however be used if the template is used in conjunction with the New-AppVSequencerPackage PowerShell cmdlet. It will support the use of all of the template items. The use of PowerShell with templates opens the door of many possibilities for automating the sequencing of your packages. Here is an example:

 

Once the package has been created, you can verify the configuration held by observing the information in the App-V manifests.

 

Happy Automation!!

App-V 5: Capturing Data Source Configuration

January 11, 2014 4 comments

You may have noticed that in previously releases of Softgrid and App-V, it was recommended as a best practice to manually create a dummy (I hate that word – I prefer “artificial”) ODBC data source (DSN) connection on the sequencing prior to sequencing anything that interacts with ADO/ODBC. The reason for this has to do with how these settings get captured during the sequencing process and how default registry opacity is handled. If the settings for ODBC are empty than any settings that are captured during the sequencing will be reconciled as “Override” since there will be no existing ODBC registry keys on the sequencer.

The problem this creates is the override settings will prevent the virtual application from seeing any existing ODBC settings that may exist on the client workstation. Given that the application may need to interface with a database client or middleware that needs these settings, this could be problematic. If you create a dummy (as it has been called) ODBC connection in advance, then only the settings for the connection parameters (DSN) captured during sequencing will be collected and the opacity will be set to “Merge.”

With the advent of the 4.6 SP1, many sequencing best practices were implemented into the workflow of the sequencer software. Among them was this very creation of an artificial DSN. As a result, people stopped having to worry about it. Well . . .

It is time to start doing this again. With the 5.0 Sequencer (including the latest SP2 release) you will need to create an artificial ODBC connection in advance of sequencing. Otherwise, you will still run into registry opacity issues with ODBC settings that are captured during sequencing. In most cases, virtual machines are used so I would simply add this to the default configuration of your base sequencing machine.

Categories: Uncategorized Tags: , , , , ,

App-V: Process Exit Codes and why they matter when troubleshooting virtual applications

May 14, 2013 3 comments

In a previous blog post (http://blogs.technet.com/b/gladiatormsft/archive/2013/04/27/on-the-art-of-troubleshooting-app-v-applications.aspx) I discussed the “art” of troubleshooting virtual applications and mentioned the fact that there are a lot of custom recipe’s out on the Internet (many courtesy of our own MVP’s.) People ask me how they figure these out. My answer is usually along the lines of “they troubleshot/revered engineered/mangled though until they came up with something that worked.” We know the next question: How do you learn to do that? To keep my promise in helping people to *LEARN* to troubleshoot virtual applications, I have been working on scenarios using real-world applications. I figured a good place to start was using an application that fails and leaves NO error message whatsoever. This is a good way to demonstrate the use of Process Monitor to track process exit codes in order to resolve issues.

The application we will be looking at is Google Chrome 15 sequenced with App-V 5. I chose this one because it is really easy to create a bad sequence (using the Google Enterprise Installer.) All you do is during monitoring – is accept all defaults (click – next – next – next etc.) Now if you ever have had to sequence Google Chrome and did your research on the Internets, you have found many complicated recipes for Google Chrome. My favorite is Aaron Parker’s (an APP-V MVP) 4.6 recipe: http://stealthpuppy.com/virtualisation/sequencing-google-chrome-15/  (Yes, I am a fan of the stealth puppy.) It is one of the many examples of one of our solid MVP’s sharing quality APP-V packaging information to prevent others from having to reinvent the wheel.

If you would like to follow along and try this at home, download the Google Chrome 15 Enterprise Installer (http://www.google.com/chrome/eula.html?msi=true) – sequence it accepting all defaults on a clean sequencer template (default exclusions) and enjoy the troubleshooting!

Symptom: I double-clicked on the virtual application – AND NOTHING HAPPENS!

So this is the issue. You have virtualized an application(Google Chrome 15) with App-V 5 and it adds/publishes successfully. However, when you go to launch the application, nothing happens. You may notice the process starts in Task Manager but then disappears. A not so nice silent exit with NO error message whatsoever. Now is a good time to break out Process Monitor and capture a simple reproduction of the launch and subsequent silent exit.

Process Exit Codes

So whenever you get strange exits or “lack of launch” with no error – your best recourse is to try to find out why those key processes exited. This time we are going to take advantage of the fact that Process Monitor tracks exit codes.

First, inventory the EXE’s with Chrome. You can do it with a local install or view the shortcut to the EXE to get the main executable. (In 4.6, you would look in the OSD file.) The key EXE’s I find with Google Chrome are CHROME.EXE and GOOGLEUPDATE.EXE.

Now, capture the issue with Process Monitor. Once you have captured the session, you will want to apply a display filter for the following operations:

–       Process Start

–       Process Exit

Apply the filter and start from the beginning. Well look! Not a lot of events! This is much easier to walk through.

The WMIPRVSE.exe is noise (WMI background process) but notice that in addition to the CHROME.EXE process, there is a “GOOGLEUPDATE.EXE” process. What is interesting is that the shortcut I clicked on was supposed to load the CHROME.EXE process. This will cause me to want to look at Parent PIDs to see what spawned what because for some strange reason, the GOOGLEUPDATE.EXE process started before the CHROME.EXE process. The chrome.exe process was spawned by Parent PID 832 – which turns out to be EXPLORER.EXE. GOOGLEUPATE.EXE appears to have been spawned by PID 1620 which is the AppVClient.exe process.

 

Looking at the command details for GoogleUpdate.exe, it begins to make sense – GoogleUpdate is a service process (GoogleUpdate.exe/svc) so it was captured as a virtual service therefore it must start before the application does. One of the many reasons why the App-V virtualization engine hooks CreateProcess.

 

You can add the “Create Process” operation to your filter and you can see Process Create events which is an easier way of looking at who spawned who.

 

In the detail we see Exit Status codes from both processes when they exit. If the exit status is 0, the process exited normally. If the exit status is a simple number, you can often look these up with the NET HELPMSG command.

I usually work these from the bottom up. Chrome.EXE exited with a status code of 6. 

 

That is an invalid handle. The GoogleUpdate.exe process exited with a numerical representation of an HREF (which is better to understand once converted to hex.)

 

Convert -2147024894 to hex and you get 0x80070002. That’s a well-known HREF

HINT – download this conversion tool – http://www.microsoft.com/en-us/download/details.aspx?id=985

It says Exchange but it will resolve standard Windows API HREF codes as well. The above resolves to ERROR_FILE_NOT_FOUND. At this point, it is time to reset the filter and create a new filter where we include the process name GOOGLEUPDATE.exe and exclude success. Then exclude NAME_NOT_FOUND
(noise.) Look for PATH_NOT_FOUND. Well what do you know? There it is!

 

And now we have to reset the filter to make sure there were no additional success events (due to path searching and alternate location searches.)

Looking at the last one registering a PATH_NOT_FOUND, unlike the previous ones where additional searches found the intended file, the attempt to create several files beneath the update folder did not. Verifying this in the package directory, we see the file is not there. By default, on a local install, the file is not there. However upon first launch when the Google Update service runs the files are downloaded successfully.

What are your Options Now?

At this point, there are a few paths we can take:

Add in those missing files? Adding in those missing files will generate the same results because there is a check from the application to download the most up-to-date binaries for the update engine. Process Monitor revealed the same ERROR_FILE_NOT_FOUND exit code.

Disable Virtual Services? You can disable virtual services altogether in either the deployment_config.xml file (for the application) or you can disable the virtual services in the registry (quick way is [reg add HKLMSoftwareMicrosoftAppVSubsystemDisable /v “Virtual Services /t REG_DWORD /d 1 ) client-wide.

 

NOTE:Upon disabling virtual services – the ISSUE STILL REMAINED. Again since the symptom is the same – I go back to the exit codes with process Monitor. This time, the GOOGLEUPDATE.EXE process did not start as desired.

Yet this time chrome still failed because of the bad handle issue. So simply taking the update component out of the sequence is not good enough. You need to be able to configure the application not to auto-update (if possible.) This goes back to the “knowing of your application” principle.

Application Disabling/Sequencing Exclusion of the Particular Service

If disabling the service through the App-V subsystem is not successful, you can still virtualize the application with App-V if there is a mechanism within the application to disengage the service altogether. To be honest, this is the best way, otherwise, the application will treat the service as a dependency for functionality. In the case of Google Chrome, you can disable this through the following registry key:

Path: HKEY_LOCAL_MACHINESOFTWAREPoliciesGoogleUpdate

Value: AutoUpdateCheckPeriodMinutes

Data Type: REG_SZ

Data: 0

In addition you will want to add in a directoryVFS exclusion for the [{ProgramFiles}]GoogleUpdate directory.

So I hope this was able to shed some light on how you get from a default-installation-based failed sequence to a working one through the use of a good reverse engineering tool. Of course, don’t forget to share your success stories in the form of recipes. You can do this on the App-V Sequencing Recipe forum here: http://social.technet.microsoft.com/Forums/en-US/prescriptiveguidance/threads

Categories: Uncategorized Tags: , , ,

On the Art of Troubleshooting App-V Applications

April 27, 2013 4 comments

Lately I have been dealing with the big challenge of teaching something that I have become to know as more of an art than a science. That is, the troubleshooting of virtual applications not functioning correctly in App-V. I have seen many articles that walk through how someone was able to figure out why one particular application was failing and how they were able to provide a resolution. We always appreciate these brave pioneers who prevent us from having to reinvent the wheel when we happen to have the same exact application problem. Through a quick search of Bing, we were able to find that hero who would rescue us from our current plight. The App-V community is full of these heroes – especially in the MVP space. What we do not know in many cases is how they went down that particular path of troubleshooting. What we will often only see (and I myself am quite guilty of this) is the abbreviated cleaned-up version of how the problem was resolved. We do not see the many paths traversed or the rationale behind the particular troubleshooting approach used.

How do you learn to do that?

I got asked this question bluntly by a customer. I was helping them remediate an application and they came right out and asked me. This got me thinking. What about the packager who is new to App-V? How does one learn how to fix a broken virtualized application, or at the very, be able to determine the actual root cause as to why the application was failing? Where does one begin when troubleshooting a broken application? What are the best tools to use?

Before we can answer these questions, we need to explore why a lot of people struggle with this. What makes troubleshooting a virtual application challenging is the lack of source codes, symbols, and more importantly, an application SME. Every time you troubleshoot a virtual application for the first time, you are basically having to learn the very depths of the application. I used to work escalation-level support and many of the rules of troubleshooting and reverse engineering that I employed applied to virtual applications. But it is one thing to support a broken application when you have every developmental resource at your disposal because you happen to work for the company that makes the very software you are troubleshooting. It is another monumental task to have to isolate and troubleshoot one that is from another vendor. Fortunately, the scope of App-V applications are limited to applications that run on windows-based operating systems.

To start, let us talk about how to avoid rabbit holes. We need to focus first on what not to do. The reasons people struggle and often never resolve application issues is usually tied to one or more of the following rabbit holes:

  • They apply a uniform . . . . rote . . . rather a “checklist” mentality where they run through the same 10-15 things to try that may possibly resolve the issue (just like purchasing a bunch of lottery tickets may land you some cash.) But when it does not work – you are back at square one and you now have several hours of your life you will never get back.
  • They assume that a method that they discovered on the Internet (and used successfully) to troubleshoot one application successfully will work on others. After all, each problematic application is the same, right? No, each application is like a snowflake – no two are alike.
  • They troubleshoot only the error and not the context of the error. While true, the error is the first clue, just keying the error into Bing and looking at the first entry may possibly lead you down a rabbit hole.
  • They assume the more they learn about one particular tool, the better they will be at troubleshooting virtual applications. The fastest way to be unproductive with Process Monitor is to apply a rudimentary blanket universal approach every time you use it. You will head straight down a rabbit hole.
  • The use the wrong tool to try to find/fix the wrong problem. Sometimes Process Monitor may not be the right tool.

 

Speaking of Process Monitor . . .

So let’s look at troubleshooting virtual applications using Process Monitor. How do you learn how to effectively do it? In my opinion, it boils down to combining several areas of knowledge –

Foundation: This is knowledge of the platform. In order to understand how applications break, you need to understand how applications work. In this case, we are dealing with Windows-based applications. A strong foundation of knowledge can be obtained by reading up on the design and internals of the windows operating system, the Win32 API, the native NT API, etc. all help to build this foundation. You do not necessarily have to know everything there is to know about the windows operating system, but having a good understanding and the ability to reference development references will suffice. Even if you are not a programmer by trade, reading the Windows Internals book (sometimes several times) will open your eyes to how critical elements such as processes, threads, and objects work. The Windows Internals book (by Mark Russinovich, David Solomon, and Alex Ionescu) is now in its 6th edition and has been split into two volumes due to its mammoth size. In addition, you have several references online that you can research and reference. The Windows API is online at MSDN (http://msdn.microsoft.com/en-us/library/cc433218%28VS.85%29.aspx) and the Windows Drivers Kit (not needed as much for virtual applications) is also online at the WDK site (http://msdn.microsoft.com/en-us/windows/hardware/gg487428 for free. You can even have an offline reference for the Win32 and .NET API’s through the downloadable SDK’s (Windows 7 and .NET 4 here at http://www.microsoft.com/en-us/download/details.aspx?id=8279)

Operational: This is knowledge of the tool. Assume the example is Process Monitor, you have to be able to efficiently leverage the right feature when needed. Since Process Monitor is the most commonly used tool used in troubleshooting virtual applications, it is essential that you understand all of the features. For any Sysinternals Tool, I would recommend the Sysinternals Administrator’s Reference by Mark Russinovich and Aaron Margosis. In the case of Process Monitor, everything useful regarding the tool is discussed. Once you know how the tool is used, you can then proceed to apply its uses to the right task.

Application Context: This is the context of the failure within the application. You cannot teach a tool if you do not know the right context of the tool especially when the use of the tool will vary from issue to issue. The application that yields an “UNKNOWN ERROR” must be troubleshot differently from an application that performs poorly, crashes, freezes, disappears without a trace, etc. I find this to be the most important skillset – the “X” factor, or so-called “Art” of troubleshooting. Those who are great at this tend to know context and thus, are able to isolate quicker.

You may find this overwhelming at first . . .

 

 

. . . and it will take practice and repetition. But like any “art” the improvement will come and you will begin to develop your own style.

Sequencing is not Virtualization and Virtualization is not Sequencing

A final note: There are big differences between “sequencing problems” and problems that are actually virtualization issues where an application is not functioning as expected. Often virtualization problems are labeled as “sequencing problems” when in fact, no problems occurred during sequencing. The problem occurred *AFTER* sequencing. Now it is true that the resolution to the virtualization problem may result in a modification to the sequencing process (i.e. a custom recipe.) True “sequencing” issues rear their ugly heads during the actual sequencing of an application.

App-V 4.6SP1: Issue with Sequencing the Dynamics NAV 2009 R2 Client

September 9, 2011 3 comments

There is an issue that currently being investigated by the App-V support team regarding virtualization of the Microsoft Dynamics NAV 2009 R2 client with the new App-V 4.6 SP1 sequencer. It appears the issue is tied to the Classic Client more and not the Role-Tailored client. In addition to the workflow being slightly different from the workflow outlined here:

http://msdn.microsoft.com/en-us/library/gg670188.aspx

The Role-Tailored client will also need to be sequenced using a VFS path rather than an SFT mount (MNT) path. You will need to change this part of your recipe:

Section: “To run Microsoft Dynamics NAV 2009 R2 Setup”

In Step 5: Do not specify the destination directory. Use the default directory.

It is also recommended to install the  Visual C++ 2008 SP1 run-time module on the Sequencer in advance.

Package Accelerators

There may be a maximum path limitation being hit preventing the creation of a package accelerator resulting from a VFS installation. This is caused by 3 factors: 1.) Lengthy paths created by the Dynamics installation. 2.) The package accelerator creation process adding additional 7 characters to the package root. 3.)  If the sequencing machine is a single volume machine where the Q: drive is actually a substitution.

You can work around this by creating a second partition for the Q: drive on the sequencing workstation.