Over the past 20 years, I have had many different roles in IT. I’ve been a helpdesk jockey, professor/instructor, sysadmin, developer, support engineer, escalation engineer, and now consultant. I’ve worked with a variety of industries as well. I’ve been both a customer and an employee of a Fortune 100 software company. As I have moved into various roles through my career, I’ve simultaneously watched the growth of the IT community in pontificating in various mediums ranging from community forums to full-blown tabloid tech journalism. I’ve learned what kind of statements garner respect and attention and what are often dismissed as hyperbole or sensationalism.
The bloggers are supposed to represent the users and/or IT pros – the “pulse” of the community. In many cases the quality of the bloggers are positive as they derived excellent content and insights due to one or more of the following factors:
Experience: A blogger will likely be taken seriously if they have the experience to back up what they are talking about. This is why the best insights often come buried deep inside of community forums and not necessarily on the site of a full-time blogger or tech journal. Why? Because blogging is not their job. They ARE an IT Pro. Blogging is merely a hobby.
Depth of Analytical Thought: They demonstrate an outstanding aptitude for critical thinking. Even if the source is focused towards a specific vendor (or as many say – biased) the analysis is spot on.
Depth of Technical Thought: Simply – they know the technology inside and out. They yield a wealth of technical information and for that reason alone, they often command respect.
I am here to tell you the influence bloggers have on software vendors and products often depend on how they engage and embrace the community around the vendor and its products – regardless of how they may “bash” a product or feature or “praise” it. If the community respects the blogger, their stature increases with the software vendor. If the blogger is simply ranting or spilling out hyperbole for the sole purpose of “click-bait,” that can come back to haunt them. This is often a challenge for full-time bloggers who are often selling advertisements to generate revenue or perhaps are freelancing for a journal who pays them literally by the click.
When you build up that large amount of overhead you need to keep those clicks and ad views going, the blogger has no choice but to be a provocateur to remain relevant in the IT tabloid media that those same bloggers helped to create. When an IT analyst or an IT research firm publishes opinions or assessments, they are always taken more seriously as they represent a wealth of combined experiences and knowledge bases. They approach product, technology, and industry analysis in a much more scientific and data-driven process. The research firms publish both the analytical and the technical depth in every case.
Since most major software vendors, at least in the US, are publicly traded, it is Wall Street that ultimately has the most influence on its direction. in IT, your shareholders are often your customers as well.
I’d been wanting to write an article on this subject for a while, but this week, I was inspired to the write this article after reading three distinct articles relating to RDS/VDI –a technology I worked in extensively. I have the unique opportunity to cite examples of an attempt of influence by a blogger, a group of analysts, and a group of investors in a very busy week for the VDI industry.
http://www.brianmadden.com/blogs/brianmadden/archive/2015/06/10/server-os-based-vdi-is-an-official-quot-feature-quot-of-windows-server-2016-apparently-microsoft-plans-to-continue-screwing-us-for-years-to-come.aspx – Basically, Brian Madden still hates how Microsoft does VDI. In other news, the Sun came out this morning.
The Analysts: http://www.gartner.com/document/3072722 – a brutally honest assessment by Gartner on why VDI is not ready for the cloud and what it will take to get VDI to a true cloud-based DaaS (Desktop as a Service.)
The Investors: http://www.valuewalk.com/2015/06/citrix-systems-inc-ctxs-elliott-associates-letter/ The investment Group Elliott Management reveal their desires for change at Citrix (the leader in VDI) in an open letter to its CEO and Board of Directors.
Which of those three articles that I mentioned do I pay the most attention to? Well, I always trust analysis over hyperbole – but money trumps all.
Since the dawn of the 21st century (and even before) you have been hearing many items related to acronyms interchangeably describing manageability features within Microsoft products (as well as others.) For example, WMI has been at the heart of most Microsoft Manageability products and solutions given the fact it is one of the primary interfaces within the Windows operating systems. While Microsoft’s WMI ties mostly to its products, it is based upon a series of open, universal standards. And this is the heart of deciphering how acronyms and standards can be interchangeably used to describe the same entity.
So let’s weave through the sometimes confusing relationship between these manageability acronyms – WBEM, WMI, CIM, DMI, DTMF, WSMAN, WinRM, and SNMP of protocols/interfaces/standards. In this little game, I will try to go through these acronyms within the average blog post attention span. WMI is Microsoft’s implementation of the open Web-Based Enterprise Management (WBEM), which comes from the Distributed Management Task Force (an industry organization.) WBEM relies on protocols – which can come from legacy standards such as RPC (Remote-Procedure Call) or DCOM (Distributed Component Object Model) or more modernized http-based SOAP standards such as WinRM (Windows Remote Management) based on the WS-MAN (Web-Services Management) standard. SOAP (Simple Object Access Protocol) itself, is is a command extension protocol designed to be used with HTTP (Hyper-Text Transport protocol – or the web) or SMTP (Simple Mail Transport Protocol – or internet email.)
The WMI interface – based upon the WBEM standard – is built upon an infrastructure centered upon the Common Information Model (CIM) and its respective Object Manager (CIMOM), is what links management applications and providers. The infrastructure also serves as the object-class store and, in many cases, as the storage manager for persistent object properties. WMI implements the store, or repository, as an on-disk database named the CIMOM Object Repository. As part of its infrastructure, WMI supports several APIs through which management applications access object data and providers supply data and class definitions.
Beyond WMI, WBEM’s architecture extends to a variety of underlying technologies besides WMI and Win32 because not everything is or will always be on Microsoft technologies – including the Desktop Management Interface (DMI), and the Simple Network Management Protocol (SNMP) Some of these standards define data storage schemas as well as interfaces. Some define commands within communication protocols. Some or more modernized. SNMP has been deprecated in the most recent versions of Windows in favor of technologies such as WinRM.
I like to use the relationship of WinRM and WMI (alongside their open counterpart standards WS-MAN and WBEM) by stating that one is a management protocol and one is a management interface.
To Read more, check out the standards themselves:
When I worked in support, I troubleshot WMI quite bit using many tools. One tool I still keep my eye on with regards to ongoing development was – and still is – the WMI Explorer utility. I am happy to report a new version of an excellent troubleshooting tool for WMI is now available:
WMI Explorer 2.0 is now available for download:
Microsoft .NET Framework 4.0 Full or .NET Framework 4.5.1
Minimum display resolution: 1024×768
Administrator rights to view some WMI objects
(Optional) Internet access for automatic update check
This is a very intuitive tool for visually troubleshooting WMI issues. It gives you a direct view into the WMI namespace.
New Features include:
New: Asynchronous mode for enumeration of classes and instances in the background.
New: Method execution.
New: SMS (System Center Configuration Manager) Mode.
New: Property tab showing properties of selected class.
New: Input & Output parameter information in Methods tab with Help information.
New: List View output mode for Query Results.
New: Update Notifications when a new version of WMI Explorer is available.
New: Connect to multiple computers at the same time.
New: Quick Filter for Classes and Instances.
New: User Preferences.
New: View WMI Provider Process Information.
Improved: UI display on higher scaling levels and resolution.
Improved: Connect As option to provide alternate credentials.
Improved: Display of embedded object names in Property Grid.
NOTE: This is not an official Microsoft tool, and is available “AS IS” with NO support.
For sequencing in App-V 5, the new ETW model simplifies the process and moves App-V to the Windows standards for event tracing. Even better, the sequencer not only has two logs to worry about (operational and administrative) but a simple process can occur to enable more verbose debug logging.
In App-V 4.6, the process was not that simple. While the logs did not write to the Event Viewer-able logs, all but one are text-based which makes for easy manipulation with your favorite log parser. I prefer Trace32 of course! These log files are stored in the logs subdirectory of the Sequencer installation directory which defaults to C:\Program Files\ Program Files\Microsoft Application Virtualization Sequencer\Logs. Certain logs pertain to specific functions so the relevancy will vary on whatever your troubleshooting scenario might be.
SFT-Seq-log.txt: The majority of sequencer logging occurs here (Uploads to virtual environment, downloads from the virtual environment, service starts and service stops, etc.)
SFTrbt.txt: This is the sequencer reboot log file. When the 4.6 sequencer simulates reboots, the elements that are processed will be tracked in this log file.
SFTCallBack.txt: This is a more simple logs that allows you to reconsile process starts and stops during sequencing. It works great in conjunction with a process monitor log.
Filter.log: Outside of working with Microsoft Support, this log is not very useful as it is obfuscated. It tracks file activity but must be decoded with an internal utility. You can enable further tracking into a file called files.txt which will contain a log of all files created in the VFS. This can be enabled (although it will increase sequencing time) by enabling the following value in the registry:
- Key: HKEY_LOCAL_MACHINE\Software\Microsoft\SoftGrid\4.5\Sequencer\Configuration\
- Value: FileManifest
- Data Type: REG_DWORD
- Data: 1
SFTrpc.txt: This is the log file created by the monitoring element SFTRPC.EXE and in addition to also capturing process startup and shutdowb, will also contain verbose diagnostic information about each monitored shortcut.
In addition to the sequencer logs, you can also leverage process monitor (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) and verbose MSI logging (https://madvirtualizer.wordpress.com/2014/03/17/enabling-advanced-windows-installer-logging/) if you encounter errors within the application during sequencing.
App-V 4.6 is still very prevalent out there and will be for a while. With the releases of Windows 8 and Windows 8.1 brought additional service packs for the App-V 4.6 client which means upgrades and/new installs for newer operating systems. Several weeks back on this blog, I went over how to enable advanced MSI logging for troubleshooting MSI installs and upgrades (Remember VOICEWARMUP – https://madvirtualizer.wordpress.com/2014/03/17/enabling-advanced-windows-installer-logging/ ) but I would like to now address some follow up emails I received. Admins would like more specific information on how to go through and read that potentially enormous log in order to find out what is failing where and when.
I would never advising reading a verbose MSI installation log from start to finish especially when dealing with potentially asynchronous actions. MSI logs also have an excessive amount of rollback information incorporated into the log upon failed installations. The seasoned IT Pro often looks for specific keywords such as “error” and “failed” and that can be misleading as not all logs generate these types of messages. In addition, searching on the string “error” can also yield false positives as well.
When I am looking at Verbose MSI logs of App-V 4.6 client installs, I usually analyze the log by doing the following:
- Searching for the error string generated in the App-V Installer User Interface with quotations.
- Searching using the string “1603.” See if it indicates that a custom action has failed.
- Searching using the string “Value 3.” This will indicate an install error. This can also help to identify the custom action failure.
- Searching for string “IsInBadState()” can also be helpful if there is an issue with a failed driver install. This is especially useful in troubleshooting an upgrade. Usually when this occurs, you usually need to delete the driver configuration and state of the specific App-V file system driver specified in order to reattempt the upgrade.
Finally if you need to walk through the App-V custom actions, you can do so by searching by the strings ‘SWI” or “SGC” as all of the App-V custom actions begin with these prefixes.
CustomAction SWI41sp1UpgradeFix returned actual error code 1603
You can walk through the logging of each key App-V custom action. Once you’ve identified what custom action failed, you can then use the following reference to find out specifically what was being attempted with the custom action here: http://support.microsoft.com/kb/2465574. Even though it specifies SP1, it is still valid and helpful for SP2 and SP3. For example the action reference above would be:
Installer : Client
Method name: SWI41sp1UpgradeFix
Description: Modifies an installed instance of the Softgrid 4.1SP1/4.2 client application to correctly support upgrading to a later version.
You can then dive deeper into the timeline of the action and align it with a more deep logging utility such as Process Monitor.
App-V 4.5 and 4.6 virtualize at the user mode layer. One of the most identifying factors of seeing that a thread stack is that of a virtualized application is the presence of the SFTLDR.DLL file. This is what is injected into every process a virtual application will create. This file is responsible for ensuring proper redirections and translations necessary to make virtualization function properly by:
- File changes to included virtual directory and file paths are redirected to the VFS
- Registry changes hooked and redirected to the virtual registry
- The spoofing of objects
- The spoofing of COM GUIDS
In addition to the common troubleshooting methods such as disabling local interaction and disabling object spoofing, you can also take things further by disabling various virtualization components using the System Guard Overrides in App-V 4.x. These are not meant to be solutions but isolation factors in case you need to modify mappings. Many of these can be set at the registry level affecting the entire client or at the application level using the OSD file.
All of the registry values mentioned are located under HKLM\SOFTWARE\Microsoft\SoftGrid\4.5\SystemGuard\Overrides:
Disabling Virtual Services
You can disable virtual services on a per package basis by adding in the <VIRTUAL_SERVICES_DISABLED> tag under the <POLICY> XML element in the OSD file. You can disable the subsystem for the entire client by going adding the DisableVirtualServices DWORD value with a value of 1. If this is enabled, the sftldr.dll will not hook the service APIs.
Disabling the Virtual Registry
You can disable the virtual registry on a per package basis by adding in the <VIRTUAL_REGISTRY_DISABLED> tag under the <POLICY> XML element in the OSD file. You can disable the subsystem for the entire client by going adding the DisableVreg DWORD value with a value of 1. If this is enabled, the sftldr.dll will not hook the virtual registry calls.
Disabling the Virtual File System
You can disable the virtual file system on a per package basis by adding in the <VIRTUAL_FILE_SYSTEM_DISABLED> tag under the <POLICY> XML element in the OSD file. You can disable the subsystem for the entire client by going adding the DisableVFS DWORD value with a value of 1. If this is enabled, the sftldr.dll will not hook virtual file system calls.
Finally, if you are really interested in going to the extreme . . .
You can disable ALL hooking. Can be useful when you are launching an application that is locally installed but still being brought into the virtual bubble. This allows you to turn it on and off if troubleshooting odd behavior. This is done at the client level which is why it is definitely only a troubleshooting option. You can disable hooking by adding in the registry value DisableSftldr DWORD value with a value of 42. Why 42? Well because that is the answer to everything in the universe. This basically makes the sftldr.dll (which is the primary hook DLL) dormant. MAVINJECT32 (or MAVINJECT64 if a 64-bit system) will still inject this DLL though. It will just remain dormant. This is a last resort.