The Default Block Size of 64KB in App-V 4.6 and Later may Cause Slow Streaming with Certain Network Configurations

Starting with the 4.6 sequencer, the default block size for sequenced packages was changed to 64K. In addition, the option to adjust this was removed from the sequencer. The block size used to be an issue when the network bandwidth was limited and large blocks could not be transferred. Now with more robust networks in place, this is not a problem anymore.

With certain 4.6 client configurations (using packages sequenced on 4.6 sequencers and later) involving RTSP streaming from management servers or streaming servers, users may notice significantly longer RTSP streaming times.

If you want to set the block size lower than 64K with the App-V 4.6 sequencer, you can still do this via the command-line sequencer. This will require an installation program, script, and/or batch file that will run completely unattended. The command line parameters are found here:

http://technet.microsoft.com/en-us/library/cc843675.aspx

You can use the /BLOCKSIZE option to specify a block size parameter of less than 64 (i.e. 32.)

To rectify this issue post-sequencing, you will need to make the following adjustments on the App-V Management server and/or streaming servers when using RTSP:

1.)    If using Windows Server 2003 for your App-V Management server, you can modify the TCP/IP settings on the Windows Server 2003 App-V Management server to immediately acknowledge incoming TCP segments per the following KB article: “Slow performance occurs when you copy data to a TCP server by using a Windows Sockets API program” (http://support.microsoft.com/kb/823764).

2.)    Use an alternative protocol for streaming (HTTP or SMB.)

3.)    Use a third-party sequencer/packager/encoder to adjust the block size.

4.)    Some of our customers have had success with adjusting TCP optimizations. For example, on servers running Windows Server 2003 Service Pack 2, you can turn on the TCP optimization feature “Receive-Side-Scaling” on by enabling the following registry parameter:

HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters

Data Type: DWORD

Value: 1

NOTE: This is only one part of the SNP (Scalable Network Pack features) that needs to be turned on. Other features such as offloading or TCP chimney do not need to be enabled. Please be aware of the fact that other applications that may be running on that server may not be tuned well for TCP optimizations. You will also need to reboot the server for this to take effect.

5.)    For servers running Windows Server 2008 and later, it is recommended to also have the receive window auto-tuning level set to either normal or experimental in addition to RSS being enabled.

To enable TCP Auto-Tuning and RSS:

• From an elevated command prompt, run the following command:

                netsh interface tcp set global rss=enabled autotuninglevel=normal

• Reboot the server.

 

App-V: Intermittent DC Refresh and/or Launch Failures may be tied to Intermittent SQL Connectivity

DC Refreshes and Application Launches may fail intermittently in the case of an unstable SQL Server connection. This can happen when the threshold for connectivity to overall service connectivity to SQL has not been reached but the authentication and/or authorization of a user actually timed out on the back end connection to SQL.
In essence, overall connections to the App-V server may still be maintained and the clients will not go into DO (Disconnected Operations Mode.) Authentications may fail due to intermittent availability of a remote SQL server.

While some have recommended modifying connection timeouts to the SQL datastore, but in my opinion, this would not be addressing the underlying issue. If connectivity is problematic to SQL, first find the root cause of the connectivity.

The error that commonly appears in this scenario is:

“A Network operation did not Complete in Time”
Error: xxxxxxxx-xxxxxxx0a-10000005

I have also seen this issue pop up when the back end SQL Server’s CPU Utilization was maxed out at 100% making the service temporarily unavailable.

App-V: Arithmetic Overflow Error (0000B065) when Running System Utilization Report

Administrators can use the System Utilization Report to graph the total daily system usage. You can use this report to determine the load on your App-V Server.

This report tracks the usage over time during the reporting period for the specified server or for the server group.

The System Utilization Report also graphs the following system usage:

• Usage by day of the week
• Usage by hour of the day

The System Utilization Report also includes a summary of the total system usage for specific users and total session counts.

There is an issue, however that you may run into when you try to run a System Utilization Report by session count where you may get the following error:

Error: 0000B065

The SFTMMC.LOG (Management Console log) will also show the following:

2011-10-14 04:35:31                  https://steveth-appv/
ManagementConsole.MCException: Arithmetic overflow error converting expression to data type int. —> SoftGrid.Management.GetReportDataFailedException: Arithmetic overflow error converting expression to data type int. —> System.Data.OleDb.OleDbException: Arithmetic overflow error converting expression to data type int.
   at System.Data.OleDb.OleDbDataReader.ProcessResults(OleDbHResult hr)
   at System.Data.OleDb.OleDbDataReader.NextResult()
   at SoftGrid.Management.DataAccess.ReportDataAccess.GetSystemUtilizationBySessionData(DateTime startTime, DateTime endTime, Int32 serverGroupID, Int32 serverID, RptSystemDailySessions[]& dailySessions, RptSystemDayOfWeekSessions[]& dayOfWeekSessions, RptSystemHourOfDaySessions[]& hourOfDaySessions, RptSystemSessionUsageSummary& usageSummary)
   at SoftGrid.Management.Reports.GetSystemUtilizationBySessionData(DateTime startTime, DateTime endTime, Int32 serverGroupID, Int32 serverID, RptSystemDailySessions[]& dailySessions, RptSystemDayOfWeekSessions[]& dayOfWeekSessions, RptSystemHourOfDaySessions[]& hourOfDaySessions, RptSystemSessionUsageSummary& usageSummary)
   — End of inner exception stack trace —

Server stack trace:
   at SoftGrid.Management.Reports.GetSystemUtilizationBySessionData(DateTime startTime, DateTime endTime, Int32 serverGroupID, Int32 serverID, RptSystemDailySessions[]& dailySessions, RptSystemDayOfWeekSessions[]& dayOfWeekSessions, RptSystemHourOfDaySessions[]& hourOfDaySessions, RptSystemSessionUsageSummary& usageSummary)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.PrivateProcessMessage(RuntimeMethodHandle md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg, Int32 methodPtr, Boolean fExecuteInContext)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at SoftGrid.Management.IReports.GetSystemUtilizationBySessionData(DateTime startTime, DateTime endTime, Int32 serverGroupID, Int32 serverID, RptSystemDailySessions[]& dailySessions, RptSystemDayOfWeekSessions[]& dayOfWeekSessions, RptSystemHourOfDaySessions[]& hourOfDaySessions, RptSystemSessionUsageSummary& usageSummary)
   at ManagementConsole.ManagementSession.GetSystemUtilizationBySessionData(DateTime startTime, DateTime endTime, Int32 serverGroupID, Int32 serverID, RptSystemDailySessions[]& dailySessions, RptSystemDayOfWeekSessions[]& dayOfWeekSessions, RptSystemHourOfDaySessions[]& hourOfDaySessions, RptSystemSessionUsageSummary& usageSummary)
   — End of inner exception stack trace —

This is caused by the system exceeding a limit of calculation. When running the system utilization report the system calculates the total session duration in seconds for all users for all of the applications that have been used during the specified time period. This total must not exceed the current limit of 2,147 million seconds.

Reduce the time period used to generate the system utilization report so it falls within the current allowed limit. 30 Days will usually be a safe maximum.

How to Adjust/increase/reduce the RTCP/RTP Port Ranges for use with RTSP

To stream applications via RTSP, the App-V server uses three channels that are carried through three TCP sockets. First, the App-V client uses the RTSP channel to set up a connection with the App-V Management or Streaming Server. The server opens two private ports (one for the RTP channel and one for the RTCP channel.) The server also sends the port numbers to the client in a response. The client then opens two sockets. Then, the client connects to the private ports that are created on the server. The application is then streamed over the RTP channel. The RTCP channel provides real-time control over the data channel.

For each application that is streamed to an App-V client, a new set of RTP and RTCP channels are created. However, all streamed applications use one common RTSP channel. The RTSP channel is also used to discover available applications on the App-V server.

The default port range is from 49152 to 65535. This can be adjusted on either the streaming server (lightweight) or the Management server (heavyweight) but each one uses different configuration sources.

RtpTcpMinPort

This defines the lowminimumstart range for the private ports.  This will be used for both RTP and RTCP. For the App-V Streaming Server, it is found in the registry at the following location:

Key: HKEY_LOCAL_MACHINESOFTWAREMicrosoftSoftGrid4.5DistributionServer

Value: RtpTcpMinPort

Data Type: DWORD

The default value is 49152 but the accepted ranges are from 1,025 – 65535.

For the App-V Management Server, this configuration comes from the SQL App-V Datastore. The database table that stores this is SOFTGRID_SERVERS and the column is rtp_min_port. The default value is 49152 but the accepted ranges are from 1,025 – 65535.

RtpTcpMaxPort

This defines the highmaximumstop range for the private ports.  This will be used for both RTP and RTCP. For the App-V Streaming Server, it is found in the registry at the following location:

Key: HKEY_LOCAL_MACHINESOFTWAREMicrosoftSoftGrid4.5DistributionServer

Value: RtpTcpMaxPort

Data Type: DWORD

The default value is 65535 but the accepted ranges are from 1,025 – 65535.

For the App-V Management Server, this configuration comes from the SQL App-V Datastore. The database table that stores this is SOFTGRID_SERVERS  and the column is rtp_max_port. The default value is 65535 but the accepted ranges are from 1,025 – 65535.

Installation Options on the Streaming Server

Both RTPTCPMINPORT and RTPTCPMAXPORT are available command line options during the installation of the App-V Streaming Server. These options are not available on the App-V Management Server installer.

App-V 4.5: Failed to Start App-V Management Server: Error 268480357

Here is the scene. An attempt to start App-V Management Server 4.5 fails with the following error:

“Windows could not start the Application Virtualization Management Server on “servername.” For more information, review the System Event Log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 268480357.”

You also then see the following in the SFT-SERVER.LOG file when it is set to VERBOSE:

[2010-07-10 15:23:21.909] - 4020 4080 SW_MessageHandler::Open - - - - 5 65535 "Initialization complete."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::MapError - - - - 5 65535 "Got unknown error code: 5701."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::HandleError - - - - 5 65535 "Error 0x1645, State: 01000, Text: [Microsoft][ODBC SQL Server Driver][SQL Server]Changed database context to APPVIRT."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::MapError - - - - 5 65535 "Got unknown error code: 5703."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::HandleError - - - - 5 65535 "Error 0x1647, State: 01000, Text: [Microsoft][ODBC SQL Server Driver][SQL Server]Changed language setting to us_english."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::HandleError - - - - 5 65535 "Error 0x0, State: 01004, Text: [Microsoft][ODBC SQL Server Driver]String data, right truncation"
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::MapError - - - - 5 65535 "Got unknown error code: 229."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::HandleError - - - - 5 65535 "Error 0xe5, State: 42000, Text: [Microsoft][ODBC SQL Server Driver][SQL Server]The SELECT permission was denied on the object 'SYSTEM_OPTIONS', database APPVIRT, schema 'dbo'."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::MapError - - - - 5 65535 "Got unknown error code: 16945."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::HandleError - - - - 5 65535 "Error 0x4231, State: 42000, Text: [Microsoft][ODBC SQL Server Driver][SQL Server]The cursor was not declared."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::SearchInternal - - - - 5 65535 "Failed to execute statement. [SELECT * FROM SYSTEM_OPTIONS]"
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SQLDataConnection::GetRecord - - - - 5 65535 "No records found."
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_DataAccess::Initialize - - - - 5 65535 "Unable to retrieve value for system options record"
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SystemDispatcher::init - - - - 1 44901 "System dispatcher initialization error [-1]. System dispatcher startup will stop.
"
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SystemDispatcher::fini - - - - 5 65535 "Shutting down System Dispatcher version 4.5.1.15580 (4020)"
[2010-07-10 15:23:22.018] APPV01 4020 4080 SW_SystemDispatcher::fini - - - - 0 44952 "Successfully shut down Microsoft System Center Application Virtualization Management Server Version 4.5.1.15580 (4020)
"
[2010-07-10 15:23:22.018] APPV01 4020 1860 SW_MessageHandler::Close - - - - 5 65535 "Shutdown complete."

You will see this occur often when you try to point the 4.5 server (Application Virtualization Management Server) to a database created by a lower version. In this particular case, the 4.5 server was trying to connect to a database previously used by a 4.1.3.16 server.

This often happens in scenarios where a 4.5 server cannot connect for some reason to a 4.5 datastore and an older copy of a previous version of the database maybe restored in error or out of desperation.

 

Regardless, this error will happen immediately upon startup. If you point to a pre-4.5 database during installation, however, the installation program will try to update it.

The best solution is to either stand up a new installation that creates a new installation or uninstall the App-V 4.5 server and install a 4.1 version of the Softgrid Virtual Application server and point to this database. Then proceed to attempt to upgrade the 4.1 server to 4.5 which in turn should upgrade the database from 4.1 to 4.5.

PLEASE NOTE: This is predicated on the fact the database is in a usable state.

App-V: 4.5 SP2 Server Fails to Connect to Failover Database Instance

Beginning with 4.5 Service Pack 2, the Microsoft App-V server now provides support for SQL Server mirroring. This allows for datastore redundancy via mirror failover in an enterprise environment. It also allows you to switch from native OS ADO over the SQL native client 10.0.

Information on how to configure this can be found here:

http://technet.microsoft.com/en-us/library/ff660790(printer).aspx

However, many have noticed that in spite of following these instructions, they were still having issues with the App-V server connecting to the failover instance.

When this failure occurs, you will see this in the server’s SFT-SERVER.LOG file:

[2011-4-07 16:33:29.212] APPVSERVER 2600 4100 SW_SQLDataConnection::PopulateConnectionString – – – – 5 65535 “Constructed ODBC connection string: DRIVER={SQL Server};DATABASE={APPVIRT};Network={DBMSSOCN};Server={APPVSQLM2,1433};”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_SQLDataConnection::MapError – – – – 5 65535 “Got unknown error code: 4060.”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_SQLDataConnection::HandleError – – – – 5 65535 “Error 0xfdc, State: 42000, Text: [Microsoft][ODBC SQL Server Driver][SQL Server]Cannot open database “APPVIRT” requested by the login. The login failed.”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_SQLDataConnection::Open – – – – 5 65535 “Failed to load requested driver.”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_SQLDataConnection::Open – – – – 5 65535 “Unable to open ODBC data source.”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_DataConnectionPool::Open – – – – 5 65535 “Failed to open connection.”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_DataConnectionPool::Close – – – – 5 65535 “Close () called on closed object.”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_DataAccess::Initialize – – – – 5 65535 “Failed to open configuration pool.”
[2011-4-07 16:33:29.259] APPVSERVER 2600 4100 SW_SystemDispatcher::init – – – – 1 44901 “System dispatcher initialization error [-1].

You will also see the following in the SQL ERRLOG file:

2010-12-29 13:55:27.18 Logon       Error: 18456, Severity: 14, State: 38.
2010-12-29 13:55:27.18 Logon       Login failed for user ‘domain\machineaccount$’. Reason: Failed to open the explicitly specified database. [CLIENT: x.x.x.x]

Cause

Microsoft had a small but serious DOC bug that took me doing an iDNA trace to find. Truth be told, I over-troubleshot it as a simple Process Monitor would have revealed it.  The SQL Failover registry keys defined here:http://technet.microsoft.com/en-us/library/ff660790.aspxwere previously listed as:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Softgrid\4.5\Server\SQLFailOverServerName

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Softgrid\4.5\Server\SQLFailOverServerPort

and we are told to create them in Steps 7 and 8.

Problem is the value SQLFailOverServerName should be SQLFailoverServerName.  That one lower case letter was preventing auto failover from occurring.

Change the registry value to SQLFailoverServerName. Since this happened, the document has been modified. The alternate languages versions of the document are also in the process of being corrected.

Steve Thomas

App-V 4.5: The App-V Management Server Service fails to start with Service-Specific Error 268480320

You may get the following error message when you try to start the App-V Management Server service:

“Windows could not start the Application Virtualization Management Server on Local Computer. For more information, review the System Event Log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 268480320

System Event Log will show the same error under Event ID 7024.

You will also see the following error in the SFT-SERVER.LOG file on the App-V Management server:

SW_ProtocolServer::ConfigureLogPipeline – – – – 1 44864 “Unexpected Error creating Log Pipeline.

This relates to the Provider Pipeline Logging Configuration which is configured at the Server Group Level.

You may also see a false positive log message stating that the server was unable to connect to the data source. The server is able to connect, but the database records it is reading in for the pipeline (to the MESSAGE_LOG table) is incorrect.

To resolve and prevent this issue, it will require correct log pipeline configuration. This is done as follows:

1.) Launch the Application Virtualization Management Console.

2.) From the Server Groups node, right-click the Server Group your App-V Server belongs to and select “Properties.”

3.) Click on the “Logging” tab.

4.) Select “SQL Database” and select “Edit.”

5.) Make necessary modifications under these considerations:

– If the SQL Server you are connecting to contains multiple instances, you will need to select “Dynamically Determine Port.”

– If the SQL Server you are connecting to resides on the default instance, select port 1433 and disable “Dynamically Determine Port.”

– If the SQL Server you are connecting to is using a non-standard port, specify the port and disable “Dynamically Determine Port.”

App-V: “A network error occurred” Error Message xxxxxxx-xxxxxx08-10000003

After upgrading to the Application Virtualization 4.5 Management Server (Service Pack 1 or 2), you may get the following error when streaming applications via RTSP:

A network error occurred error :  4524284-15603808-10000003

 

You will also get this error if the installation is a clean installation.  Publishing/Refresh still works but streaming applications fails.

This can occur if the content path is set incorrectly in the registry. This is likely caused by an incorrect value set during the installation. Please note that this error is different from previous versions and does not yield the normal 1F4 or 194 error.

Solution is to correct the value for the content folder under:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SoftGrid\4.5\Server

Value: SOFTGRID_CONTENT_DIR