App-V 5.0 Sp3 Upgrade: Clear Browser Cache to clear issue on Server MC upgrade

Just in case this bites someone else…

An upgrade of App-V Server to 5.0 SP3 includes a replacement Management Console.  As this is Silverlight based, this means that the website delivers a Silverlight XAP file to your browser.  Where it remains in your browser cache.

Upgrade of the server does not always mean that the browser cache gets flushed.  So you might find that while your console is “working”, you are not getting the new UI features (in this case for connection groups). To verify, open the console and click on the “About” link (upper right) to check the version, which should now be 5.0.10107.  If not, clear your browser cache.  And don’t forget about any shortcut links that you made to the console.  In my case, typing the URL in a browser got me the new console, but clicking the link each time got me the old one.


Tim on App-V 5.0 SP3

image attribution Stuart MilesThis week Microsoft released the anticipated App-V 5.0 SP3, along with an updated UE-V 2.1, as part of the MDOP fall release.  We normally expect the MDOP release around November 1st, so this drop is a little later than usual for some reason.  But it is well worth the wait.

While Microsoft App-V is generally regarded as the best application virtualization solution out there, the product did take a step back in the 5.0 re-write that was released two years ago.  Oh, it was still better than the alternatives, but not necessarily better than App-V 4.6.  At that time, I cautioned companies to work with 5.0 in their labs, as eventually they will want to move to the new platform, but if they were on 4.6 currently they shouldn’t be in a hurry to upgrade in production. 

With the release of 5.0 SP2 last fall, I felt that it was mostly ready.  On one hand, 5.0 SP2 allowed you to do many things that you just couldn’t do otherwise, while on the other hand there were some annoyances that kept it from being declared the best.

During 2014, we saw a couple of “hotfix” releases, especially HotFix 4 and HotFix 5, which were really feature releases and major performance improvements.  5.0 was looking really good.

With the release of SP3 this week, I think that 5.0 is now the best release, clearly surpassing even App-V 4.6.  Which is important since the end of support for App-V 4.6 SP3 as I read it as Dec 2 2014,  or this guy thinks  might be next June.   In any case, time to get moving. 

So what do we get in SP3?

To start with, everything here is based upon a combination of access to pre-release software, plus a review of the release notes.  The msdn download center has been all bonkers so I haven’t gotten the actual bits yet. 

In the release, all components of App-V are upgraded.  Server, Sequencer, and Client.  There are MDOP downloads, RDS downloads, and Hosting Provider downloads showing on the site (even if the download button isn’t working.

See the release note here, but read my summary as well…

  • Single Install. First, we get a roll-up of those important “hotfixes”.  A single install for new clients is a beautiful thing.
  • Connection Group Flexibility. This consists of four things:
    1. You can now include apps published to the user and those published globally in the same group (as long as you enable the connection group to the user).  Previously all packages in the group had to be published the same way as the group.
    2. You can mark packages as optional in the group.  This means you can define a master group for anyone getting the primary package that includes all possible add-ons, but publish the add-ons independently without having to manage a group for each permutation.  It is kind of like the And/Or used in the Configuration Manager, except that the group GUID is consistent at the client.
    3. You can mark a package version with a wildcard of “*”.  So if you upgrade the package you don’t have to think about updating the connection group.
    4. You can now unpublish and/or remove a package from the client without disabling the connection group (as long as the package is optional in the group).
    5. Connection groups can get upgraded even if in use.  This is similar to the package pending scenarios (where a package add/publish/unpublish/remove is queued up for execution at a later time when the package is not in use), except that in this case pending means that the client needs to make additional changes to state but you immediately get the effect for newly launched packages/groups.

Keep in mind that this new flexibility is not available to Config Manager deployments at this time.  That would require a rev of Config Manager.  So those that deliver using the App-V Server, Stand-alone deployments, or with third-party tools, get a leg up.

  • Publishing for other users. You may now publish/unpublish a package, or enable/disable a connection group to a user other than yourself via powershell.  The limitation is that the user must be currently logged in, or at least profile still available on this machine, and that you must be an administrator.  The powershell now accepts the user SID as an optional parameter.
  • Local Publishing Authorization.  The Group Policy (as well as regular registry based configuration) for App-V is expanded to allow you to control if package and connection group publishing is limited to only Administrators.  Previously, it was possible for users with standard rights to self-publish by googling the PoweShell commands.
  • Merged Roots. This is huge! When packages are placed in connection groups, an improved merging of identical paths between the packages occurs.  There were improvements in HotFix 5, but is even better now.
  • Goodbye PVAD (almost). Thanks to merged roots, it is no longer necessary to declare a PVAD folder in the sequencer and install the software to be virtualized into that folder.  In fact, I found out that if you are going to use a Connection Group, you are probably better off using a VFS style install! With SP3, there still is a PVAD, it is just hidden.  For most all applications, this turns out to be just fine, but we know of a few applications that require PVAD installation, so how to we handle that?  You have a couple of ways to do this:
    1. There is a new command line switch, -EnablePVADControl when you launch the Sequencer GUI to show the PAD prompt like before.
    2. You can set a new registry DWORD HKLM\Software\Microsoft\AppV|Sequencer\Compatibility\EnablePVADControl to 1.
    3. Or, when you enter monitoring mode, just browse to the C:\ folder.  The Sequencer created a random PVAD folder name (looks like a GUID) for you.  Just install the app there.

So how do you know when to sequence to the PVAD?  .  Each app will act differently, but often the when at the client and you launch the virtual app, it will in some way complain that it isn’t installed or configured correctly and refuse to run. When the app doesn’t work at the client, give PVADing a try.  It might just work. 

The MVPs have a short list of apps that we have found that need PVADing, including:

ArcGIS Desktop tool
Avaya CMS Supervisor R17
Wonderware InTouch 10.x

I also think that we will find apps that have long path length issues with VFS style installs, and a PVAD install, especially one carefully crafted with a long PVAD, might be needed.

  • PowerShell Changes. Some small changes were made to the PowerShell interface to support the new features.
  • Client Publishing from Publishing Server. Some behind-the scenes changes were made to support the new connection group features.  This mostly affects manual diagnostic debugging from the client of the publishing information as the new client will automatically use the new syntax.  If previously you used http://servername:port to test the publishing server, you now use http://servername:port&clientversion=5.0.xxxxx.x&clientos-WindowsClient_6.2_x64 (replacing the x’s with the actual client version and the OS as well).
  • RunVirtual. RunVirtual is now supported for per-user published packages. WooHoo.
  • Some App-V Client Debug logs have been consolidated, and some moved to a ServiceLog subfolder.  See the release notes if you care.
  • Virtual Services with DACLS Fix. For some time, certain windows services that use “unusual” Service DACL settings to protect themselves would not work when installed using the sequencer.  This included the FlexNet licensing service and those of Google Chrome Enterprise.  While the prior work-around to manually modify these permission settings in the Sequencer will still work, you no longer need to do so.

NOTE: Microsoft doesn’t list this as a feature or fix with the release.  It seems that this was simply a side effect of a different fix to virtualized services.  They only became aware of this fix when I discovered it and asked about it.  No word on what they were trying to fix, but presumably it was to fix a different virtual service issue reported by a customer.

Please pay attention to the details in the readme about the order of upgrade installation, as this is different from prior releases due to the changes to support the new connection groups.  You should upgrade the servers before upgrading ANY clients.

This information is based on working with some pre-release builds, and re-checking the release notes to make sure that there are no unexpected surprises. We will switch out and use the new version in our training class this week (if I can get it downloaded!).  Perhaps we will learn a little more. 

Also check out Thamim’s blog on the subject here:

PS: I have a new version of AppV_Manage ready to release that works with the new features, and works with the Windows 10 Preview, where there are some PowerShell differences that needed addressing.  I will get that released soon.

App-V and .Net Performance

This post is about the performance impacts of combining Microsoft App-V 5 sequenced virtual applications with applications that use Microsoft .Net.  I wish this story ended with good news, or at least reasonable actions that you could take to improve performance, but it isn’t so.  Instead it is mostly informational, however, in the end there is a technique mentioned that might be useful for a really poorly performing app. In the most typical cases the trick is probably too much work for the performance gains that you might receive, but maybe it can help you if you are stuck with user’s complaining about the runtime performance.  So here is the story (so far).

Background on .Net

Like Java, Microsoft .Net programs are constructed with the concept of “write once, run anywhere”.  Except anywhere means places where you have the supporting runtime.  For Java, this means the appropriate Java Runtime (which can be placed about everywhere), and with .Net it means the appropriate .Net Framework (more limited currently, but now it might be coming to other OSs).  But while Java may run on more CPU architectures and OS platforms, .Net tends to have much fewer version incompatibilities. Both programing languages achieve processor independence by compiling into a processor independent language.   This is called MSIL (Microsoft Independent Language) in “.Net-speak”.

The processor independent language is compiled into binary form by a “just-in-time” compiler, known as JITing. Both Java and .Net use JIT compilation to improve speed over interpreted languages like the older Visual Basic (the non-managed variety) or Perl languages. While binary code will always be faster than interpretation, the overhead of JIT compiling every time the program is run is a drag on performance. Microsoft supports three forms of performance improvements for .Net programs that the developer of the app can take advantage of.

  1. Strong Named Assemblies. When a .Net dll is added to a system, if it was built with a StrongName it can load faster.  A StrongName is basically a weak public digital signature that doesn’t verify authenticity but verifies that the file hasn’t been messed with.  When dlls built with StrongNames are stored in the Global Assembly Cache, signature verification occurs only once (when placed into the cache).  When the dll is loaded by a program into memory, the OS can perform a quick hash check that the file hasn’t been modified since installation into the cache.  For large dlls where you might not need all of the functionality this will improve performance by a small amount.  To gain this performance, the StrongNamed dll must be added to the .Net Global Assembly Cache (one of the following folders under C:\Windows\assembly: GAC, CAG_32, GAC_64, GAC_MSIL).  The dll placed in the Global Assembly Cache is not compiled, so gains are limited. Some .Net dlls are built without a StrongName, and these will never be added to the Global Assembly Cache.  Adding this into the cache is typically a post-installation step (sometimes performed by the installer).
  2. Native Compilation (post-install). When a .Net dll or exe is installed on a system, NGEN can be used to create a “Native Image” copy via compilation.  This compiled copy is specific to a CPU architecture.  Back when I attended the “Hailstorm” event in Redmond (when .Net was originally announced more than a dozen years ago), compilation to a CPU was very processor specific.  But these days, my sources inside Microsoft inform me that we don’t need to worry about AMD versus Intel, or Xeon versus Core I*.  Compilation is really to the bit-ness of the CPU architecture (32 versus 64 bit) and major design.  Usually, these native compilations occur as part of the application installer as a post-installation event.  Often, the installer designer forgets about this, and the program is just JITed all of the time.  Post-install compilation is performed using the ngen.exe program that is part of the framework.  The compilation may be performed synchronously, or placed into a queue for asynchronous compilation by the NGEN service whenever the system is idle for a period of time.  When compiled, an executable named foo.exe or bar.dll will have a compiled copy placed in the windows\assembly subfolder for the major .Net version (2 or 4, currently).  These files are distinguished by being called or  These files will almost always be larger than the original file, but usually perform much faster as JIT compilation is no longer needed at runtime.  The system loader automatically detects if there is a Native Image available whenever the original component is loaded into memory.
  3. Native Image Compilation (by the developer).  This is a new option available to developers in Visual Studio 2014 preview for applications being developed for the Windows Store.  Because the compilation is being performed at build time, the optimizations available are far greater.  Among other things, this compilation pulls in the portions of framework dlls directly into the native image (sort of like embedding mfc dlls into a Win32 executable), making these Native Images independent of the .Net Framework itself.  Of course on the flip side, these Native Images are processor specific. The result is a real exe, so special handling by the loader is not required.

.Net compilation can improve performance significantly.  There are no hard numbers on this, as it really depends on the program.  Some simple tests I have run using post-installation compilation into Native Image on small programs saw a 15 to 20% improvement in launch time.  So the potential performance gains of .Net Native Image compilations are significant enough to be interested in.

Which brings us to the .Net and App-V discussion.

When we install and capture programs in the sequencer, post-installation activity of type 1 and 2 are a possibility and may get captured.  If captured, we are essentially redistributing these components to the clients.  I used to think that this would break things, but never saw that happen.  And now with the latest information from Microsoft, it seems not to be the case and redistribution in an App-V sense should be OK.

To confirm when a native image is in use, I use ProcessExplorer to examine what is actually loaded into the process memory. Using the lower pane display to show loaded modules, you can see that both AppV_Manage.exe  and are loaded in the process shown in the image below:

Process Explorer showing a process using a Native Image

Note: The above capture is with AppV_Manage version 3.8 where I realized that my installer wasn’t creating the native image either.  You will now also see the compression library has a native image also.

In tests not using App-V, I have proven the ability to redistribute an “AnyCPU” executable that was natively compiled on an x86 system.  Outside of App-V, I redistributed the native image to both an x86 system and an x64 system.  The x86 target system used the Native Image (as seen in ProcessExplorer) and saw the expected performance gains.  The x64 system did not use the Native Image compiled for 32-bit as the “AnyCPU” executable ran as 64-bit on that system, thus the performance was the same as if not compiled.  But it wasn’t any for the worse!  So the bottom line is that capturing the native image and redistributing might make things better and don’t seem to make things worse.

But it turns out that we don’t often capture native images in our App-V packages.  I recently added support to the AppV_Manage Analyzer (version 3.8) to detect the number of modules in the Global Assembly Caches and the Native Image caches.  An example display is shown below:

AppV_Manage Analyzer showing a package containing a Native Image

Why don’t typically we capture Native Images in our App-V packages?

There at least a couple of reasons:

  1. The installer didn’t request the compilation. Depending on the installer, this may require the addition of a custom action.
  2. The installer requested asynchronous compilation.  This makes the installer seem faster, but unless you let the sequencer sit around a while after the installation is complete it won’t happen in time.  Pretty much, unless you are interrupted or go get some coffee or a smoke, the compilation won’t happen in time before you end monitoring.

So I was thinking that I’d like to force the compilation.  Either case can be handled by using ngen.exe which comes with the framework.  You might have two copies, one for v2 and one for v4.  When in doubt, use the v4 copy (it handles .Net 2 apps also).  To solve the second issue, you only need to run the command:

C:\Windows\Microsoft.Net\Framework\v4.0.30319\ngen.exe ExecuteQueuedItems

On a 64-bit system, you should also run the command in the Framework64 folder.

To solve the first issue, you identify the major exe programs and run:

C:\Windows\Microsoft.Net\Framework\v4.0.30319\ngen.exe Install "C:\...pathto...exe"

If you compile the exe, it automatically compiles the dependent dlls, so typically you do not need to compile the dll files directly.  Do these things while in monitoring mode, and the global and native image caches are automatically captured.

But does it do any good to do this in your packages?

Unfortunately, NO.

Currently (as of App-V 5.0 SP2 with Hotfix 5 on a Windows 7/8.1 system with .Net 4 through 4.5.2), the loader cannot detect the native images that are inside the App-V package. Nothing is broken, it is just that the VFS caches for .Net are not used.  Some simple testing indicates that it is not a timing issue.  There is an undocumented index file which might play a role in this, but I think it is a deeper issue.

Potential Workaround?

In theory, if you had a really badly performing virtual application, you could use a Publish-Package script to force an ngen compilation against the major exe(s).  You would have to create the package using the forced load for the streaming configuration, and then run this script outside of the virtual environment using the ProgramData reference to the exe.  If you had a really horrible app, it might be worth a try.  Keep in mind that you might not be able to have multiple versions of the same app with improved performance with different Native Images if done this way – it depends on the app.

I am reporting this into Microsoft for consideration.  So the story could have a happy ending one day after all.  Maybe.


I should mention that while I only tested Microsoft App-V 5.0 SP2 (with Hotfix 5) in preparation of this article, it probably applies to any technology used to virtualize or layer applications.  Ultimately, I believe any filter driver, or user-mode dll injection, that adds application components to the OS will fail to deliver the native components in a way that the OS will use as intended.  This includes other application virtualization tools (like Symantec or VMWare ThinApp), and application layering solutions (like UniDesk, and VMWare AppVolumes, but maybe not FsLogix which should work as is).  At least with App-V I know that we can work around it, but I’m guessing that due to their architectures, some of the others cannot.


Don’t let your App-V 5 Reporting Database grow out of control!

Image from Sql Management StudioThe App-V Reporting database for App-V 5 and above is great for monitoring all usage of your virtual applications.  This is quite helpful when it comes to license compliance.  While it won’t prevent (accidental) usage of unlicensed apps, it is very useful for detecting usage as part of a program to “true up” your license allocations.

The Reporting Server is independent of how you deploy the virtual apps.  It doesn’t matter whether you use the App-V Server System Center Configuration Manager, or deploy using the stand-alone client.  You can still use the reporting server.

If you do, I highly recommend that you use the App-V Client GPO to configure the clients to use the reporting server.  Setting this as an enforceable policy makes sure that every client will report in usage data each night.  Only with non-persistent clients (like some VDI clients or PVS booted XenApp servers) is there a concern of not getting the data, but those can be solved by adding a report sync to the user logoff scripts using the App-V Client Powershell API.

There is always a but…

But this blog post is to warn you about a small flaw in the App-V Reporting Database setup.  By default, it will not clean out old data.  EVER!  I am not referring to the temporary table data (removed when records get consolidated each night), but the record data on each and every virtual application run by a user. The App-V 4.* server would automatically clean this out at some point.  But not version 5. While Microsoft added logic in the form of stored procedures to clean old data out, it isn’t used by default.

The stored procedure, spEnforceSizeLimit, is designed to check the size of the current database and compare it to a configurable maximum size. There are also two additional configured settings, a low water mark and high water mark.  These are stored as percentage factors and are interpreted as relative to the configurable max size. So the current value can be in one of thee states against these settings:

  1.  If the current size of the database is below the low water mark it decides that things are OK. If logs a WARNING message to the Sql database log (“WARNING The database size limit cannot be enforced“)  and exits. I guess it is a warning that maybe you aren’t collecting any data and maybe you should check on that.
  2. When above the low water mark and below the high mark , it will seek to purge the oldest application usage data.  The logic is simple (much simpler than the extensive logic used in the original App-V server) but sufficient.  It will take a ratio of the current size to that of the low water mark, look at the date of the record about that far into the usage table (when sorted by date) and remove anything older than that date.  It adds a message to the Sql database log (“WARNING: The database size limit has been enforced“).  This message is a good thing and indicates that the purging is working.
  3. When above the high water mark, it logs a different message to the Sql database log (“WARNING: The database is cleaning up a large percentage of data“), and then processes just as in the previous case.

So here are the two problems in the default install preventing this purging from working:

  1. It depends on the maximum size, and low and high water marks being present in the SystemOptions table of the database, and these are not set by default, as shown in the image below.Image from Sql Management StudioThis is treated by the stored procedure as an error, however no error message is logged and no purging occurs. This means that the stored procedure will do nothing when called of there isn’t a row in this table.
  2. There is nothing that triggers the stored procedure to run anyway.Image from Sql Management Studio
    The image above shows the default jobs in the database, one to handle the daily processing of received client reports, and the other a standard database job. No job to run the cleanup script though.

And I wouldn’t be writing this if there wasn’t a solution…

To solve these issues and ensure that your database doesn’t infinitely grow (well, or until your DBA notices), the DBA should do the following:

  1. Ensure that Sql Server Agent is running on the database server.  (If it isn’t, the stored procedure to process the client data from the temporary tables isn’t being triggered to run and you won’t get reporting data anyway).
  2. Add a row to the empty SystemOptions table.  Set the max_db_size column with a value for the maximum database size in Megabytes.  Each record in the Application Usage table takes around 200bytes of storage.  Multiply that by the number of users, then by the number of average app launches per day, then by the number of days of data you want to keep.  Then add some overhead for the temporary and other tables (maybe 50MB?) and spare capacity and you have your setting for the maximum size.  Set the low and high water percentage mark appropriately, but be aware that in spite of the name they are really factors, so 0.80 and 0.90 might be good numbers.  Make sure to monitor the logs after a while.  The warnings I mentioned provide details on the numbers calculated to help you adjust these settings.
  3. Add a SQL job to run that calls the spEnforceSizeLimit stored procedure.   There is already a job to call the spProcessClientReport stored procedure once a day, so mimic that job.
  4. Play the logic game on when things happen, and possibly adjust timers.
    • The App-V GPO schedules the reports to be uploaded, typically once a day at night.  The upload from a client is fairly quick, but there is a configured random delay when the scheduled time occurs.  If the random delay is 30 minutes, then an hour after the scheduled time is plenty of time for this to complete. This should  be allowed to complete before the spProcessClientReport is scheduled.
    • The job for spProcessClientReport should be scheduled next.  The amount of time for this job depends on the number of users*applaunches for the day.   But if you assumed that it can complete in an hour you would be safe.
    • Finally, the spEnforceSizeLimit should be scheduled last.  It won’t hurt anything if it runs while either of the other two activities are happening, but it makes sense to make this last.

Now I’m pretty certain that your DBA does not follow my blog, so if you are the App-V guy and are reading this, pass it on to him or her.

App-V 5 Management Server DB on Remote Machine Configuration

Well here was a wasted hour trying to figure this one out.  It should have been so easy!

Problem: Installed a lab App-V 5 Management/Publishing Server with default instance database on that server.  Works great.  Install second Management/Publishing server needing to use the remote database.  Fails.

What could have been the problem: (but wasn’t)

  • Windows Firewall on first server with the database.  You need port 1433 open.
  • Need to give machine account for new server permissions on the database.
  • Name resolution, need to use fully qualified names, etc.
  • User login account (used the same domain admin account on both servers).
  • Remembering to reset the Management Service (iisreset) after making any change to the configuration.

What could not have been the problem: (but needed to be done anyway):

  • Using the first Management Server, add the Second server into the list of authorized publishing servers. (This only affects the rights of the publishing server to talk to the management server).

Bold face verification of things that were not the problem:

  • Ping the database server by name.
  • Install Sql Management tools on new server and connect and successfully query the database.

Actual Problem: The settings in the Windows Registry HKLM\SOFTWARE\Microsoft\App-V\Server\ManagementService were incorrect.  Furthermore, was misleading me for some reason.

That post suggests the following values (where {name} is something you substitute with the real values):

Registry Name Registry Value

In my case, using the default instance database server with the default database name on the remote server, this seemed to indicate I should use this:

Registry Name Registry Value

This did not work! Swapping in the fully qualified server names or even IP address was no help. Trying MSSQLSERVER$ did not help.

Viewing the working server, the values look like this:

Registry Name Registry Value

For these values to work, it seems that:

  • MANAGEMENT_DB_NAME must default to the default instance when not specified.
  • MANAGEMENT_DB_SQL_INSTANCE of “.” (a period character) specifies the local server (the period) and lacking a specified instance uses the default instance there.
  • MANAGEMENT_DB_SQL_INSTANCE must default to the local server when not specified.

So I tried the following:

Registry Name Registry Value

This worked!

PS: Some good links:

On Streaming and Formats in App-V

On Streaming and Formats in App-V

Tim Mangan
TMurgent Technologies, LLP
September 22, 2014
Related Product:  App-V 5.0 SP2 HF5 and prior

When we created the original application virtualization product, SoftGrid, streaming was a very important part of the implementation.  Over the years, the streaming aspect has become less important to implementations, while remaining important to the decision makers that choose application virtualization with App-V.  What some may not realize is how App-V 5 changed how streaming is done from the previously generation product.  This document identifies the differences and suggests that in most cases, streaming should not play such a large role in the decision to virtualize.

Historical Roots

Softricity, which created the original version of App-V called SoftGrid, was born out of another company called SoftwareWow.  SoftwareWow focused on Application Streaming for delivery of the applications, but did not virtualize the applications.  I joined SoftwareWow in 1999 due to my background as an expert in developing network protocols.

When we made the change to Softricity to build SoftGrid, streaming remained as one of the core properties of the product value.  We led discussions with customers based on three tenants: Centralized Delivery, Streaming, and Virtualization:

  • Back then, most software was manually installed by an IT Admin.  We didn’t have SCCM or use other Electronic Delivery Systems, with the exception of some home-grown scripting.  Install once and deliver globally was a great value proposition to the enterprise.
  • Because of our streaming background, delivery via streaming was important to us.  And we found that customers reacted extremely well to the streaming.  If they remembered nothing else of what the sales person told them, they remembered the demo.  The user doesn’t have the app, a shortcut appears, they launch the app and just the portions needed stream over and the app launches quickly.  Most competitors also implemented and featured streaming, with the exception of ThinStall (later known as ThinApp), which focused solely on the virtualization feature.
  • Solving application conflict, especially “DLL hell”, and allowing multiple versions of the same application simultaneously.

What is Streaming?

But what is “streaming”?  When viewed from the standpoint of the receiver (the virtualization client software), streaming is acting on the contents of the source (the virtual package) before receiving the entire contents.  In the case of App-V the entire contents refers to the entire App-V package file.

The concept of dynamic application delivery is more appealing when the dynamic delivery is fast.  While we can always suggest faster disks, faster networks, and faster processors, streaming is quite appealing as it allows for a more immediate consumption of the application.  Of course, in reality we usually aim to get the entire package pre-cached by pre-deploying the package before the user even asks for it.

This streaming is implemented differently in the original implementation (SoftGrid and App-V prior to 5.0) and in 5.0.  But in both cases, the streaming is implemented with an understanding that the Windows file system supports the consuming of files based on 4k file boundaries.

Streaming Prior to 5.0

Streaming with SoftGrid, and App-V prior to version 5.0, was implemented based on in two ways.

First of these was the development of a streaming source file format.  Internally known as the “Jigsaw format”, the base file format of the SFT package file included a division of internal files based on 4k file blocks, to mimic how Windows pages executable files from the local disk when running applications.  This file format, along with the file system drivers implemented at the client, allowed for the client to request individual blocks on demand.  For efficiency, the format actually packaged multiple of these blocks into a delivery chunk, and the chunk could be individually compressed for improved storage and delivery.  Originally there were 8 blocks (32kb before compression) in a chunk by default, and later 16 block chunks were used.

Second, the Jigsaw format allowed for customization of the layout of chunks within the file to optimize delivery.  We focused on optimizing the initial launch of the application, defining a “Feature Block” that would contain only those blocks needed to start the application.  This allowed for the client to make a single efficient request for the data, and because this feature block was stored contiguously on the disk, it made reading of the block when stored on rotating media more efficient also.

As a side note, we originally implemented two “feature blocks”, the first (FB1) containing the blocks to start the application, and the second (FB2) which held the remainder of the contents, which normally would be requested individually by the client on demand.  But our plans called for implementing additional feature blocks eventually.  We believed that we could monitor package use of FB2 at the client in order to define additional feature blocks that contained contents needed for a particular feature (such as the “mail merge” feature in Word).  So while never implemented, the idea was that we could have multiple of these feature blocks defined in the file format, with the final block being used as FB2 was currently used.

The package metadata included a listing of file offsets against the overall Sft file.  So if the application requested the bytes from the range 0 to 4k of DataFile2, this would be known to the client as part of Chunk N+1, which would be downloaded and decompressed if not present at the client.

This streaming could have been implemented using complete files, which is how Citrix Streaming was implemented.

SoftGrid and App-V before 5.0 also used an external XML file to control the virtual application and streaming process.  The OSD was actually a very early attempt at logically defining a software package for publishing purposes.  Originally proposed by Microsoft and Marimba in the mid-1990s, the proposal to the IETF went nowhere and was then abandoned until we picked it up and extended it for application virtualization when we built SoftGrid.  As we will see in App-V 5, the idea of a common file format to define a software package is of interest today by many technologies.

Streaming in 5.0

App-V 5.0 uses a different format to define the package.  See for a blog post introducing the format used.

This format is rooted in the Open Packaging Conventions (OPC) of ISO 29500-2 (, which is the basis of formats for Microsoft Open Office (, AppX, the package format for “Modern” applications (, and App-V (not yet formally documented).  In addition to sharing a base of ISO 29500-2 with these other formats, App-V leverages some of the XML formats used in AppX.

OPC defines a flexible container format to encapsulate a “package” that contains multiple items.  For the Open Office format, the container can represent a Word or Excel document, for example.  For AppX, the container would represent an application.  For App-V, the container represents a package of virtual applications.

OPC allows for use of the Zip format for the overall container.  Zip was originally defined by PKWare, but they made the format public in 1989 (  OPC specifies a subset of the ZIP format for usage, and imposes a number of restrictions on options that specifically may not be used.

The ZIP format consists of a header (located near the end of the file), a central directory, and a number of records.  The central directory provides the list of files, and for each file the offsets compressed into which records.  Each record is up to 64KB (pre-compressed) of an individual file. In addition to the compressed data, the record contains the full file name, attributes, and extended attributes.  Although App-V could have added its own extended attributes as part of the record,

OPC mandates the 64-bit extensions to zip, and that only the “deflate” protocol may be used for the compression. There are also a number of other restrictions, such as allowable file-names, and a prohibition to encrypting files (you could include a file that was externally encrypted, but if you use the ZIP encryption you need to get a license from PKWare).

OPC also defines, but does not require, one specially named folder in the archive (_rels).  This folder contains xml files defining relationships.  App-V does not make use of this optional part of the OPC.

Content Types

OPC defines some particular files, which generally are optional.  One required file is the “[ContentTypes].xml” file.  The ContentTypes file defines how every file should be interpreted based on the file extension or filename.  This xml file consists of “Default” elements that map each of the file extensions found in the file to a file type and subtype, and “Override” elements that map specific files.

<?xml version=”1.0″ encoding=”UTF-8″?><Types xmlns=””><Default Extension=”dat” ContentType=”appv/vfs-file” />
<Default Extension=”dll” ContentType=”appv/vfs-file” />
<Default Extension=”HxS” ContentType=”appv/vfs-file” />
<Default Extension=”hxc” ContentType=”appv/vfs-file” />
<Default Extension=”hxt” ContentType=”appv/vfs-file” />
<Default Extension=”hxk” ContentType=”appv/vfs-file” />
<Default Extension=”xml” ContentType=”appv/vfs-file” />
<Default Extension=”rll” ContentType=”appv/vfs-file” />
<Default Extension=”XLL” ContentType=”appv/vfs-file” />
<Default Extension=”XLAM” ContentType=”appv/vfs-file” />
<Default Extension=”xltx” ContentType=”appv/vfs-file” />
<Default Extension=”IDX_DLL” ContentType=”appv/vfs-file” />
<Default Extension=”XLS” ContentType=”appv/vfs-file” />
<Default Extension=”XLSX” ContentType=”appv/vfs-file” />
<Default Extension=”CHM” ContentType=”appv/vfs-file” />
<Default Extension=”DOC” ContentType=”appv/vfs-file” />
<Default Extension=”txt” ContentType=”appv/vfs-file” />
<Default Extension=”xsn” ContentType=”appv/vfs-file” />
<Default Extension=”EXE” ContentType=”appv/vfs-file” />
<Default Extension=”GRA” ContentType=”appv/vfs-file” />
<Default Extension=”FLT” ContentType=”appv/vfs-file” />
<Default Extension=”EPS” ContentType=”appv/vfs-file” />
<Default Extension=”GIF” ContentType=”appv/vfs-file” />
<Default Extension=”JPG” ContentType=”appv/vfs-file” />
<Default Extension=”PNG” ContentType=”appv/vfs-file” />
<Default Extension=”WPG” ContentType=”appv/vfs-file” />
<Default Extension=”ADM” ContentType=”appv/vfs-file” />
<Default Extension=”odc” ContentType=”appv/vfs-file” />
<Default Extension=”INI” ContentType=”appv/vfs-file” />
<Default Extension=”ICO” ContentType=”appv/vfs-file” />
<Default Extension=”TTF” ContentType=”appv/vfs-file” />
<Default Extension=”CNT” ContentType=”appv/vfs-file” />
<Default Extension=”HLP” ContentType=”appv/vfs-file” />
<Default Extension=”manifest” ContentType=”appv/vfs-file” />
<Default Extension=”HTM” ContentType=”appv/vfs-file” />
<Default Extension=”PPT” ContentType=”appv/vfs-file” />
<Default Extension=”XLA” ContentType=”appv/vfs-file” />
<Default Extension=”CNV” ContentType=”appv/vfs-file” />
<Default Extension=”DIC” ContentType=”appv/vfs-file” />
<Default Extension=”MSG” ContentType=”appv/vfs-file” />
<Default Extension=”trx_dll” ContentType=”appv/vfs-file” />
<Default Extension=”msi” ContentType=”appv/vfs-file” />
<Default Extension=”ONE” ContentType=”appv/vfs-file” />
<Default Extension=”onepkg” ContentType=”appv/vfs-file” />
<Default Extension=”H” ContentType=”appv/vfs-file” />
<Default Extension=”ECF” ContentType=”appv/vfs-file” />
<Default Extension=”FAE” ContentType=”appv/vfs-file” />
<Default Extension=”CFG” ContentType=”appv/vfs-file” />
<Default Extension=”HOL” ContentType=”appv/vfs-file” />
<Default Extension=”SAM” ContentType=”appv/vfs-file” />
<Default Extension=”OFT” ContentType=”appv/vfs-file” />
<Default Extension=”POTX” ContentType=”appv/vfs-file” />
<Default Extension=”ACL” ContentType=”appv/vfs-file” />
<Default Extension=”LNG” ContentType=”appv/vfs-file” />
<Default Extension=”LEX” ContentType=”appv/vfs-file” />
<Default Extension=”dub” ContentType=”appv/vfs-file” />
<Default Extension=”SCM” ContentType=”appv/vfs-file” />
<Default Extension=”CSS” ContentType=”appv/vfs-file” />
<Default Extension=”OPG” ContentType=”appv/vfs-file” />
<Default Extension=”dotx” ContentType=”appv/vfs-file” />
<Default Extension=”ITS” ContentType=”appv/vfs-file” />
<Default Extension=”xsl” ContentType=”appv/vfs-file” />
<Default Extension=”tlb” ContentType=”appv/vfs-file” />
<Default Extension=”rdlc” ContentType=”appv/vfs-file” />
<Default Extension=”xap” ContentType=”appv/vfs-file” />
<Default Extension=”config” ContentType=”appv/vfs-file” />
<Default Extension=”xrm-ms” ContentType=”appv/vfs-file” />
<Default Extension=”mui” ContentType=”appv/vfs-file” />
<Default Extension=”TTC” ContentType=”appv/vfs-file” />
<Default Extension=”OCX” ContentType=”appv/vfs-file” />
<Default Extension=”ORP” ContentType=”appv/vfs-file” />
<Default Extension=”PSP” ContentType=”appv/vfs-file” />
<Default Extension=”chr” ContentType=”appv/vfs-file” />
<Default Extension=”SHP” ContentType=”appv/vfs-file” />
<Default Extension=”INF” ContentType=”appv/vfs-file” />
<Default Extension=”ELM” ContentType=”appv/vfs-file” />
<Default Extension=”OLB” ContentType=”appv/vfs-file” />
<Default Extension=”man” ContentType=”appv/vfs-file” />
<Default Extension=”CAB” ContentType=”appv/vfs-file” />
<Default Extension=”eftx” ContentType=”appv/vfs-file” />
<Default Extension=”thmx” ContentType=”appv/vfs-file” />
<Default Extension=”ODF” ContentType=”appv/vfs-file” />
<Default Extension=”pri” ContentType=”appv/vfs-file” />
<Default Extension=”propdesc” ContentType=”appv/vfs-file” />
<Default Extension=”gpd” ContentType=”appv/vfs-file” />
<Default Extension=”cat” ContentType=”appv/vfs-file” />
<Default Extension=”MOF” ContentType=”appv/vfs-file” />
<Default Extension=”WAV” ContentType=”appv/vfs-file” />
<Default Extension=”kic” ContentType=”appv/vfs-file” />
<Default Extension=”CPL” ContentType=”appv/vfs-file” />
<Default Extension=”msp” ContentType=”appv/vfs-file” />
<Default Extension=”WMF” ContentType=”appv/vfs-file” />
<Default Extension=”MID” ContentType=”appv/vfs-file” />
<Default Extension=”BMP” ContentType=”appv/vfs-file” />
<Default Extension=”BDR” ContentType=”appv/vfs-file” />
<Default Extension=”ICM” ContentType=”appv/vfs-file” />
<Default Extension=”PUB” ContentType=”appv/vfs-file” />
<Default Extension=”POC” ContentType=”appv/vfs-file” />
<Default Extension=”DPV” ContentType=”appv/vfs-file” />

<Override PartName=”/AppxManifest.xml” ContentType=”application/” />
<Override PartName=”/AppxBlockMap.xml” ContentType=”application/” />



Although I am unsure if the [ContentTypes].xml file is actually used by the App-V client, the sequencer does product such a file as part of the package, which would be necessary to claim compliance with the format.  As shown in the example (from the virtual Microsoft Office 2013 package), there are only two “override” elements:

  • AppXManifest mapped to “application/”.
  • AppXBlockMap mapped to “application/”.

All of the remaining file extensions contained in the package are mapped to a general “appv/vfs-file” file type and subtype.   In this case, vfs means that it is a file in the package and does not specify PVAD versus VFS.  This default types includes the xml extension which covers the remaining xml metadata files located above the “root” folder.


The AppXBlockMap file contains information for each file in the zip archive which appears to be more about file verification.  Each file element in this xml has the uncompressed file size, and logical file header size, plus a sub-elements for each of the zip records.  The sub-element indicates the uncompressed size of the block, and a calculated hash for the block. Given that this information is largely discoverable via the central directory with the exception of the calculated hash, it seems reasonable to assume that this is used as an extra level of verification used in AppX and borrowed by App-V.


The AppXManifest is an extended version of what AppX uses to define applications.  App-V adds to the syntax defined by AppX to add virtualization specific information.  A subset of the AppXManifest file is produced external to the package by the sequencer in the form of the DeploymentConfig.xml and UserConfig.xml files.  It is part of the metadata required by the client, but not part of the streaming format.


The StreamMap.XML file is what defines the feature blocks of App-V 5.  In App-V 5, there are three feature blocks, the first two of which are roughly equivalent to FB1 of the old system.

  • Primary Block:  This block includes all file (parts) of the archive that are necessary to add the package to the client during “add-appvclientpackage”.  Certain required files (those above the “root” folder) are assumed and not listed but are automatically brought in by the client.
  • Publishing Block: This block includes file parts needed for publishing.  This will include portions of each application exe (even if not launched, the client needs the file header brought down), the icon files (for the shortcuts to look right), and any file parts launched during streaming configuration.

The files listed may contain sub-elements listing the blocks involved (the entire file is needed when no block elements are recorded).  These block elements consist of a starting block number, and the number of blocks needed.  The block number refers to which compressed block of the file (1 through N).

The remaining files and file parts are part of the third (unnamed) feature block and are streamed on demand.

If during streaming configuration in the sequencer, you select the checkbox to “Force application(s) to be fully downloaded before launching” then the StreamMap will consists of only a PrimaryBlock with an indication for full loading.


According to the Zip format, the central directory may contain records for both folders and files.  OPC, and App-V, only record files in the central directory.  The file names provide the relative path information, allowing the client to be able to extrapolate the folder objects needed in the Cache.

Unless there is a folder in your App-V package that contains no files.  While I am unsure that OPC prohibits the inclusion of directory objects in the zip central directory (the specification is a little fuzzy about that), it is not something that the Open Office Format would expect.  So the designers of the App-V file format chose to create a special XML file to hold this and other special file-system data.

The FileSystemMetaData file contains two types of XML elements.  The first lists the path for all “empty” folders.  This is why you don’t see any empty folders if you just open the AppV file using a standard zip utility (which is unaware of the special purpose of the FileSystemMetaData file).

The second element type is for defining “opaque” folders.  These are folders (whether empty or not) that are marked OverrideLocal in the virtual file-system editor.  This setting could have been added as a special attribute to a folder object in the zip archive central directory, if folders had been added.

A potentially missing element in this file would be folder “deletion markers”.  Just as registry keys and registry items can be deleted while sequencing and recorded, potentially folders and files deleted while sequencing could have been recorded.  App-V 4.* had deletion markers for both the registry and the file systems, while App-V 5 supports only registry system deletion markers.  I am unsure why this was skipped, but honestly I can’t (currently) think of an application where I require that feature.


Streaming is alive and well in App-V 5, but do you really care?  I say that streaming is important, but configuration of the streaming in a package is often irrelevant.

Most App-V implementations do not require streaming configuration.  It only affects the first time use of the application package by the user.  The addition of “background streaming” in recent years makes real-time “fault streaming” a rare event.

For most App-V environments, you either want to configuring the streaming by launching nothing (because you are using the App-V Server for publishing in a semi-persistent environment like XenApp), or use the checkbox (because you are using SCCM with BITS, aka “download and run” for delivery).

Streaming is important in non-persistent VDI, although you typically enable Shared Content Store Mode, so streaming configuration is irrelevant (everything is fault-streamed without caching).


Other Authorities………………………… 3……………………… 3…………………………………………………….. 3………… 3……………………………………………… 3


Updated solution to Paint.Net plugins in a connection group

Maybe Steve Thomas isn’t as crazy as he looks? I thought he went a little nuts when he wrote this blog post and suggested that you should always sequence using VFS. I mean, that is dead set against everything Microsoft has ever said. I did not disagree with the idea of VFS’ing things, but I have never said to VFS all of your packages. But now I’m not so sure he is all that crazy.

In February, I posted a solution to Paint.Net plugins delivered via App-V Connection Groups ( It was ugly, but instructional for solving similar problems. In our second Eastern Europe class last month, a student (let’s call him ‘C’ since I don’t have permission to use his name) finished the lab early and tried one of the things that I had said wouldn’t work. With Success!

It seems that Microsoft made a change in the Hotfixes that makes Connection Groups easier, and perhaps we didn’t really understand the small print that came with the hotfix all that well.

Ok, first of all there probably is not a great reason to use Connection Groups with Paint.Net plugins. Just put them in the main package and be done with it. But demonstrates a different sub-style of locating plugins than is normally used by an app. The primary types are:

  • Registry Key Style Browser apps, for example (Internet Explorer and others) use a registry style integration. In this style, the application supports additional plug-ins in the form of dlls. The Plug-in typically may be located anywhere on the disk, but is registered by adding a registry entry under a known app-specific key. Normally, the value of the registry item is the full path to the dll file. At app startup, the app reads all of the registry items under that key and loads the dlls into its memory space. Registry style plug-ins generally work well with App-V and Connection Groups.
  • Folder Style In this style, the application supports additional plug-ins in the form of dlls also. The Plug-in, however, must be located in a known specific folder. The application might use a hard-coded path, or a registry based string to know the location of the folder, or it might use a relative path from the current working directory, or it might do what Paint.Net does. In this style, the app looks for dlls in this folder and loads them all into its memory space. Thanks to PVAD and VFS, these styles may or may not be troublesome for App-V Connection Groups.

In the Folder Style, installing the main app to the PVAD generally caused a problem, because while you can specify the same PVAD folder for the plug-in, the PVAD folders do not overlay. Instead, the PVAD is actually moved at the client based on GUIDS, and the different packages in the connection group will have different GUIDs, such that the plug-ins are never seen.

In the folder style, installing the main app to the VFS, and plug-ins to the same VFS area seems to work when the application locates the folder using a hard-coded path, or a registry string based location. When the application uses the current working directory of the application, by default this will be rewritten as the GUID path of the primary program, but we can manipulate the current working directory to the VFS location by manually manipulating the DynamicConfiguration.xml file, and this will work. Technically, this working directory doesn’t exist until the virtual environment is open, but you can specify a fictional working directory when you launch an application so Windows doesn’t care that it isn’t present until the app starts running.

But then there is the folder sub-style that Paint.Net uses. We can’t be sure exactly what the code does, but it appears to locate the location of the executable’s exe file (probably using a programming technique referred to as reflection). And in App-V, whether or not the app is VFS’d, this location will always result in a path under the App-V cache GUIDs of the package containing the EXE. Even when the primary package is VFS’d, this meant “AppV\Guid\Guid\Root\VFS\…”. So the plug-in would not normally be seen.

My previous trick was to think like the client. This involved sequencing the main app in the PVAD, and sequence the plug-in by first installing Paint.Net exactly like it would appear at the client (under AppV\Guid\Guid\Root), and then VFS install the plug-in.

What ‘C’ found, when sequencing using 5.0 SP2 with Hotfix 4 and testing on 5.0 SP2 client with Hotfix 5, was that he could VFS install and then also VFS the plug-ins — and it worked. I know that this did not work in the original 5.0, and I am 95% sure that it did not work in 5.0 SP2 (since I posted this after the release of SP2). So it probably came in with one of the hot-fixes. So what changed?

Well, it appears that a further merging of the VFS has been added in the hotfixes, probably in Client Hotfix 5. In the past, the VFS paths only merged when the program tried to read from the overlayed location. So previously, with both packaged VFS’d, only if the app requested C:\Program Files\Paint.Net\Effects would the merged files be seen. Since the app requested “C:\ProgramData\AppV\Guid\Guid\Root\VFS\ProgramFilesX86\Paint.Net\Effects”, previously the app would only see the files of the package using those GUIDs. But now, it seems that a request to this folder is redirected to appear as the C:\Program Files\… counterpart where merging, not only with other packages in the Connection Group, but potentially with local (native) files when the parent folders are marked merge with local.

This change, along with the change to let you choose to allow the application to update data and setting files (but not executable components) located in the VFS, make it much more reasonable to sequence to the VFS.

But should you regularly sequence to the PVAD or VFS?

Microsoft Sequencing “Best Practices” say PVAD first, unless you have a problem.

Microsoft PFE MCS Steven Thomas (known as @MadVirtualizer on Twitter and “The Gladiator” when he blogs on TechNet), recently blogged that he is changing his mind and that perhaps you should always install to the VFS due to a rash of issues that he has resolved by changing a PVAD install to a VFS one. I am guessing that Steve’s post should have referenced that he assumed that you are using 5.0 SP2 HF5 (or above).

I am not sure yet whether this is a good “standard practice”. I do know, from my research papers on App-V 5.0 SP2 performance that there is a small, yet measurable, performance penalty to opening files in the VFS. I also know that I had issues with non-PVAD installs of certain apps in the past (at least before the fixes), and issues with other apps unless I VFS’d them. With App-V 4.x, I argued that VFS’d installs were generally OK and I usually defaulted to the practice unless it caused a problem. But in App-V 4.x I never found the reputed performance hit for VFS that I see in 5.0.

So I am not (yet) confident that VFS first is the wisest default, but whether you PVAD or VFS, I would recommend always starting with one, and change the practice on a per-app basis only if you encounter a problem. But stay tuned, we are sure to address this topic again as we learn more.

Errror 000012 on Add-AppvClientPackage

A customer was using XenApp with PVS. The same PVS image was used on machines on a farm. They added App-V 5 to the PVS image and decided to use the PowerShell interface to add applications virtually at boot time. One of the servers refused every app, while everything else worked on the ‘identical’ servers.

Investigation showed that the Add cmdlet was failing with Error 0000012.

No other messages in the event logs, even turning on the debug logs.

In further investigation, the customer had configured the App-V client to place the package store on a secondary partition. On boot, while Windows was wiped out, this partition remained. The folder referenced existed on both good and bad machines, but was empty on the bad machines.

Looking at permissions, since the standard Windows error 12 is an Access error, the security settings and standard attributes of the good and bad server folders were Identical.

Solution: The App-V client protects the package cache. As such, it expects to create and set special attributes on this folder and everything under it. Possibly someone manually created this folder, causing the problem. The solution was just to delete the folder referenced in the registry config. The next time an app was added, the App-V client recreated the folder correctly, and everything worked.

The long term solution was to add the deletion of this permanent cashed to the bootup script. They also could reconfigure the client to the ‘standard’ location, but reducing writes to the PVS partition is probably preferred.