To continue the subject of WMI troubleshooting: I am always frustrated at the quick, shotgun method of rebuilding the WMI repository as a rote, rudimentary troubleshooting step. This is very dangerous and risky. Rebuilding the WMI repository manually has resulted in some 3rd party products not working until reinstallation – IN SOME CASES – even this does not work. This is especially a shame since it may not always be necessary and even if it were – if there is severe WMI corruption, it may almost be better to “in-place” upgrade the OS or do a complete reinstallation of the operating system and software as incomplete repositories can yield lingering problems.
Before you go down that road, ask yourself the following:
1.) Have I properly troubleshot the error to the point that the only possible source could be corruption?
2.) Have I researched and installed all of the latest service packs and hotfixes related to WMI? (Hint: Go to http://support.microsoft.com and search WMI, hotfix, and <OS>)
3.) Have I gone through the rudimentary WMI checklist so I don’t get burned by simple things such as firewall rules? (Hint – http://madvirtualizer.wordpress.com/2014/01/22/the-importance-of-troubleshooting-wmi-part-2/)
4.) Have I ran WMIDiag (http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=7684)
and received really bizarre errors like 0×80041010 WBEM_E_INVALID_CLASS or does it fail to connect at all?
5.) Do you fail to connect to WMI Control from Computer Management?
If you answered “yes” to 4 out of the 5 above, here are some steps you can do to safely troubleshoot the WMI repository using “soft” actions:
Instructions for Operating Systems Prior to Windows Vista (XP/2003)
1. Open the Services MSC console and stop the Windows Management Instrumentation service.
2. You will be prompted to stop dependent services. Click “Yes” on the prompt.
3. Once the service is stopped, browse to %SystemRoot%\System32\wbem. Rename the Repository folder to Repository.old.
4. Once the folder has been renamed, restart the WMI and dependent services in the following order:
Windows Management Instrumentation
SMS Agent Host (If SCCM/SMS is installed)
Windows Firewall / Internet Connection Sharing (ICS)
Any other services that were detected as dependencies
5. Once services have restarted, verify the Repository folder has been recreated. Now bear in mind, it could take up to one hour for WMI to fully rebuild.
Instructions for Windows Vista and Later (2008/Win7/2008 R2/Win8)
1. Open an elevated command prompt.
2. Verify the WMI repository is not corrupt by running the following command:
If the repository is not corrupted, a “WMI Repository is consistent” message will be returned. If you get something else, go to step 3. If the repository is consistent, you need to troubleshoot more granularly. The repository is not the problem.
3. Run the following commands to repair WMI:
If the repository salvage fails to work, then run the following command to see if it resolves the issue:
After the last command, there should be a “WMI Repository has been reset” message returned that verifies the command was successful.
Even the above commands should be a last resort if you are getting an error related to “Access Denied” or “RPC Server unavailable.”
To continue my discussion regarding the importance of troubleshooting WMI, I want to move the focus to a devising a targeted approach when troubleshooting so you can optimize the time it takes for you to zero in on the issue.
WMI issues generally fall into the following areas:
Configuration Issues: These are issues relating to the configuration of WMI on the local (or mostly remote) machine including:
• DCOM Security\Permissions or Configuration
• Firewall Configuration
• WMI namespace security
Infrastructure Issues: These are issues related to WMI components including:
• WMI service setup
• DCOM registration problems
• Missing WMI classes
• Improper WMI provider registration
• Missing System files
• WMI repository corruption (*GASP*)
• Deleted WMI repository (*HEADDESK*)
WMI Managed Entity Issues: These may be issues related to the extensible WMI components including:
• Security requirements
• Not running (e.g service, application) or de-installed application
• External dependencies
As I mention in my last article, you obviously want to verify your firewall rules (which are built into versions of Windows since Windows XP.)
WMI (ASync) Properties – In
WMI (DCOM) – In
Port: TCP 135
WMI (WMI) In-Out
Then you will want to zero in on the error itself.
0x800706BA – RPC Server Unavailable
When this error appears during connecting to a WMI namespace:
• The machine does not exist.
• The machine cannot respond because the appropriate firewall exceptions have not been made. Check firewall settings.
When this error appears during operation it could be:
• The client machine doesn’t have correct firewall settings for asynchronous call backs.
• Connecting to a machine which doesn’t exist.
0×80070005 – E_ACCESS_DENIED
When this error occurs during connecting to a WMI namespace –
• The username/password does not exist.
• The user does not have the remote launch or remote activation options set.
• Check dcomcnfg.exe under the COM Security Tab.
When this error occurs during operation –
• The specific user does not have the DCOM permissions.
• Minimum authentication level needed for the namespace is more than what is used.
• Check the settings on the Default Properties tab of DCOMCNFG.EXE.
0×80041003 – WMI Access Denied
During connecting to a WMI namespace – The user does not have the appropriate WMI permissions on a namespace. Check WMIMGMT.EXE and permissions for that namespace.
During operation – Specific user doesn’t have WMI access permissions.
0×80041001 – Unknown Error
Ah, the UNKNOWN ERROR. Often this is cause by a 3rd-party provider or non-OS software that extends the Repository has been either removed from the environment and left WMI subscriptions in the repository or is malfunctioning.
Enable WMI Verbose logging on the server and review the WMI logs in %SYSTEMROOT%\system32\wbem\logs. The Wbemess log will show which WMI subscription was sending notifications when the criteria was met.
You will need to follow the steps below to remove the WMI subscriptions once you isolate them:
1. Click Start, run, type Wbemtest then type root\cimv2\applications\ and click “Connect” button
2. Click on ‘Enum Classes’, click the Recursive radio button, click OK.
3. Scroll down until you see _FilterToConsumerBinding class. Double-click on it.
4. Click the “Instances” button on the right hand side.
5. Choose those you isolated and click on the delete button.
When you retrieve a managed resource in a WMI script, the CIMOM (WMI service) looks for the managed resource’s blueprint (class definition) in the default namespace if no namespace is specified. If the CIMOM cannot find the managed-resource class definition in the default namespace, a WBEM_E_INVALID_CLASS (0×80041010) error is generated.
0x8007000E – Not enough Storage is available to complete this process
This usually indicates a problem with a provider, handle leak, memory leak, or other problem tied to WMI functionality.
1. Use the WMI Control to ensure that the service is working on the local system.
2. If the problem involves communicating with a remote system then use the WMI Control to test the ability to connect to the remote system
3. If the service appears to be working, use verbose logging to see the activity (queries) that is being processed by the service and to identify any problems. You can also use WMICHK and WMIDIAG to check the health of the service and the hosted providers.
4. For Access Denied type issues verify that the DCOM and WMI Service settings are at default values, and the Network Service account has been granted impersonation rights.
5. Check the service settings if the WMI service fails to start or if client programs cannot communicate with the service. In some cases you may need to reregister all the modules to recover the service.
6. If queries appear to be returning an incomplete results set, try increasing the buffer thresholds.
7. If problems persist, make a backup copy of the existing WMI database (repository), and then try building a new one.
Often in Virtualization and Management Products like SCVMM, MED-V, Config Manager, UE-V, and App-V the symptom of an issue appears in the respective System Center or MDOP product but the root cause is often caused by an anomaly in an underlying operating system component. Often that component is WMI. For this reason, it is invaluable have a solid understanding of WMI and WMI troubleshooting. WMI is often a component that can cause problems due to one or more of the following WMI issues:
- Corrupted repository
- Incomplete namespace
- Access Denied
- Invalid String in WMI property/data
- Unexpected value
- Memory leak
- Code Defect by WMI Provider
One of the most common errors encountered is error 0x800706BA – RPC Server Unavailable.
This error has context. If it is during connecting to a WMI namespace, it is usually because:
- The machine does not exist.
- The machine cannot respond because the appropriate firewall exceptions have not been made. Check firewall settings.
If it is during operation, it is likely because:
- The client machine doesn’t have correct firewall settings for asynchronous call backs.
- Connecting to a machine which doesn’t exist.
First I would verify the firewall rules. I would make sure the following rules are set:
- WMI (ASync) Properties – In Program: %SYSTEMROOT%\System32\WBEM\unsecapp.exe
- WMI (DCOM) – In Port: TCP 135 Program: %SYSTEMROOT%\System32\svchost.exe
- WMI (WMI) In-Out Program: %SYSTEMROOT%\System32\svchost.exe
I deal with WMI problems all the time. I generally follow this little troubleshooting checklist for RPC errors:
- Use the WMI Control MMC (WMIC.MSC) to ensure that the service is working on the local system.
- If the problem involves communicating with a remote system then use the WMI Control to test the ability to connect to the remote system
- For Access Denied type issues verify that the DCOM and WMI Service settings are at default values, and the Network Service account has been granted impersonation rights.
- Check the service settings if the WMI service fails to start or if client programs cannot communicate with the service. In some cases you may need to reregister all the modules to recover the service.
I recently had an issue where I encountered something quite bizarre. In an effort to reduce size on disk of dynamically expanding virtual hard disks (VHDs) I found myself feeling like I needed to take medication. After sysprepping an operating system image on the disk, the current file consumption on the disk was approximately 12 GB and the size of the dynamically expanding virtual hard disk was 15 GB (with a capability of growing to 127 GB as that was the size designated when creating the VHD.)
I then mounted the disk as a drive in Windows 7 using diskpart.exe in order to perform some offline defragmentation, pre-compaction, and compaction. I found that after disk defragmentation was completed in Windows 7, the total size of the disk grew to the full 127 GB although the total file consumption the disk was still at 12 GB. I had never encountered this before. I have, in the past, seen defragmentation cause some gain in VHD size but only ever at a maximum of 10-20%. To top that off, pre-compaction and compaction did nothing to reduce the size.
Now, just to give some background, the Windows 7 virtual stack used to mount VHDs did not understand the TRIM command (which is what the file system started sending down in Win7 to let the storage stack know an area was no longer in use). Anytime defragmentation is run on a dynamically expanding VHD where the stack doesn’t understand TRIM will in fact result in a larger VHD than you started with. BUT NOT THIS MUCH! I even verified that volume snapshots were disabled on the volume as that can also explain a large increase in the size of the VHD.
Realizing this was done on a host machine using a customer’s corporate operating system image, I took a copy of the original VHD and mounted and defragmented the disk on one of my plain vanilla operating system images and found the behavior I expected – only a nominal increase in size. At this point, I realized something outside the norm of the operating system was causing this growth. I could have easily done the tedious approach of removing individual 3rd-party filters on the image (using the divide-and-conquer method) while running defragmentation but I wanted to see if what was doing this was even related to defragmentation. I decided to simply just mount the drive again and monitor the disk size while doing absolutely nothing interactively to affect the drive.
I went to lunch. I came back, the disk was already at 32 GB. By the end of the afternoon, it was back to 127 GB. There was obviously some file-system based software performing this. It turns out, there is a McAfee Encryption policy in place (they were running 3rd-party disk encryption software) that silently encrypts new logical drives as they are added. When I mounted the VHD through Windows 7’s Disk again while this software had been disengaged, the issue did not occur.
I hadn’t been taking crazy pills after all.
I recently had a customer inquire further as to how the mechanics differ between all of the application modes in MED-V. I would have thought this far into the life cycle of MED-V that I had gone into enough detail on the subject. Turns out, while the article I wrote on TechNet a couple of years back (http://blogs.technet.com/b/medv/archive/2011/06/02/med-v-v2-why-would-an-application-fail-in-seamless-mode-when-it-succeeded-in-full-desktop-mode.aspx) gave a good high-level explanation, more clarification is needed.
So in addition to the information I laid out back in 2011, I’ve done some more diving into RAIL (Remote Applications Installed Locally) the inline VPC implementation of TSRemoteApp (now called RemoteApp) where the RemoteApp Server component was ported to Windows XP for the guest integration.
First of there are actually two “full-screen” modes in MED-V 2. One involves starting full screen mode from either the MED-V toolkit or using the command MEDVHOST /fullscreen to launch the workspace in full screen mode with the MEDV Guest services and agents still engaged. If you were to access the Virtual PC out of band using the VPC Window or by double-clicking on the .VMCX file, you would get the warning message about another user being logged on (see http://blogs.technet.com/b/medv/archive/2011/08/24/med-v-v2-strange-message-lt-virtual-pc-name-gt-was-closed-with-a-user-logged-on.aspx)
I outlined the basics of the differences in this high-level chart:
RDPINIT/RDPShell and Active Setup
In addition to items that depend on Explorer.exe such as Login Scripts, Active Setup also does not run. You may can get around this by leveraging group policies and the RUNONCE.EXE command. You can specify the startup applications as a part of a user’s logon settings in Group Policy. Because Group Policy controls these settings, any startup application that you specify runs as expected when the user logs on. To specify the startup applications as a part of a user’s logon settings, follow these steps:
1. In the server Group Policy Management Console (GPMC), select the GPO (that applies to the GUEST OS [XP]) and edit.
2. Click Computer Configuration, and then click Administrative Templates.
3. Click System, double-click Logon and then double-click Run these programs at user logon.
4. In the Run these programs at user logon Properties dialog box, click Enable.
5.Click Show, and then click Add.
6.Type the name of the startup application – runonce.exe /AlternateShellStartup (must include the argument)
7.Click OK two times.
One Final Note on Termination
When a Remote Application is terminated, the process on the XP Guest that is associated with the application is terminated. However, the RDP session remains in a disconnected state until is reset by an administrator. This behavior can be modified by using the group policy “Set time limit for logoff of remote app sessions.”
Any other processes that should be terminated when the Remote Application is terminated can be specified in the following Registry key.
This is a registry key that the RDP RAIL Shell uses to filter out “system” processes (in addition to rdpshell.exe and rdpinit.exe). Processes configured via this key are ignored by the RAIL Shell. In addition, they will be terminated upon exiting of the RDP session.
A process can be added by adding a REG_DWORD value with a name of the process and a value of 0×0. The following is a list of processes that are terminated by default when a Remote Application ends.
Clipsrv.exe Conime.exe Ctfmon.xe Dwm.exe Imepadsv.exe
Lmsvcs.exe Msgsvc.exe Nddeagnt.exe Netdde.exe Netstrs.exe
Os2srv.exe Proquota.xe Rdpclip.exe Screg.exe Taskeng.exe
Within MED-V V1, you can modify the default policy interval by modifying one of the XML files on the client. While the official documentation alludes to this:
“A typical MED-V Management Server can support 5000 users based on the recommended hardware configuration. The client-server communication is lightweight: clients are normally configured to poll the server for policy every 15 minutes, and image updates every 4 hours (Can be changed using MED-V configuration)”
What is does not state is how exactly you change the policy update interval from the default interval of 15 minutes.
To modify the Policy Update Interval, this would have to be done on the client side. You can do this by editing the Configuration.XML located in %ALLUSERSPROFILE%\MED-V\Local\Config
The value is UpdaterTimeInterval and it is found under the element as displayed in the example below:
<Configuration> <SpecialFileProcessor> <WorkspaceKeys type="System.String">Kidaro.KeysStorage.KeysConfigurationFileProcessor</WorkspaceKeys> <ClientPolicy type="System.String">Kidaro.Policy.Server.PolicyFileProcessor</ClientPolicy> </SpecialFileProcessor> <!-- Should the service update the configuration from the server. --> <EnableServer type="System.Boolean">true</EnableServer> <!-- Time interval in seconds between updates of configuration files. --> <UpdaterTimeInterval type="System.Int32">900</UpdaterTimeInterval> </Configuration>
Increase the default UpdaterTimeInterval value of 900 (15 minutes in seconds) to desired interval then save the file and restart the MED-V Client service.
Running a MED-V application that depends on presence may not properly show presence when hovering over it in the System Tray
Let’s review some basic information about how MED-V: The way MED-V V2 works is the Windows 7 host machine connects to the Guest Virtual PC through an RDP-style connection. This basically turns the Windows XP Virtual PC into a mini-RDP server. This must always be in the back of your mind while you test your applications under a MED-V solution. Leveraging RDP removes the need for a hooking DLL to be injected into the guest OS and cuts down on the overhead of the MED-V Guest Agent.
Since applications that run under MED-V are basically the same to the Windows 7 host as applications running remotely on an RDS or Terminal server, you will encounter specific limitations in cosmetic desktop features. For example, the AeroPeek style thumbnail preview of the remote application will not be visible. Window titles will show an appended (Remote) to differentiate it from the local applications.
In addition to what comes through the remote connections, MED-V will republish (pass along) critical messages that appear in the Windows XP system tray. For example, password expiry notices and update notices from WSUS (or Configuration Manager) will also appear on the local desktop. Applications that publish to the Windows XP System tray in the guest will also appear in the host (with an appended “Remote.”)
One item that is not simply a cosmetic issue that you will need to be aware of when considering MED-V for application remediation are applications that have presence indicators in the system tray. Changes in presence often cause a change in icon or icon color as well as their pop-out status message. While these status icons will appear in the Host system tray, there will be potential issues with changes in user presence updating icons properly. Applications such as Communicator, Windows Messenger, and Lotus SameTime may not always update/change presence notifications properly when running in a MED-V workspace.
Let’s use the example of a user being signed in initially as “available.” When the use steps away and becomes idle, the system tray icon may not initially reset the icon appearing in the host to “Away” even though the user is away from their desk.