ATM: The Cache Machine!

ATM is a new tool designed to allow you to better explore how the operating system uses the System File Cache and Standby Lists on your system. In addition to some monitoring functions, on newer OSs you may also modify your system cache behavior.

More importantly, on Vista, Windows 7, and server builds, this tool allows you to investigate prioritized memory and what Microsoft calls "SuperFetch".

Download links are at the bottom of this page.

What is the System File Cache?

The System File Cache is a component of the operating system kernel. It works in conjunction with the Memory Management sub-system and the IO sub-system to cache files, or more acuratly portions of file, in RAM. This important function significantly affects system performance by supporting sharing of commonly used files (such as system dlls) in memory, and avoids unnecessary disk IO.

What am I looking at?

ATM was designed for IT professionals, not your average user, so there is a lot stuff on the screen. But just start with the two graphs.

The upper graph provides a view of RAM (memory) usage. It is a historical graph showing the last 60 seconds with the most recent interval on the right. The two "green" categories reflect what is considered "Used Memory" and will match up with what the Task Manager reports. The dark green shows the ammount of that memory which is in the System File Cache Working Set. The other colors are the "available" memory. Not all available memory is empty, however. All but the "Free & Zero" category are memory pages that have been removed from working sets or precached and are on the standby list(s), meaning that they can be used without disk access if needed (again).

The image above was taken from a Vista OS.  If there are as many categories on your graph, you are on Vista, Windows 7, or the server builds.  On earlier operations systems, there are fewer colors (categories) shown because those versions had only one big standby list.  On these older operating systems, a dark blue represents the portion of available memory that contains parts of files which were recently in the System File Cache but are no longer in use.  If requested again (soon), the OS can recover these file pages without reading them in from the disk again.

On Vista and above some things have changed and we have more categories of "available" memory to display. 

Standby Lists.  This is pageable kernel memory that has been paged out recently, or pre-cached for potential use.   But now instead of one big standby list we have 7 prioritized lists. If a page fault occurs for this memory, it is recovered via a soft page fault as long as it remains in this list, otherwise a hard page fault is required to recover from the page file. Memory from things like indexing or background virus scans should end up in the lower priority lists. If Superfetch is anticipating file use, it would load it into higher priority of the lists. If you run out of zero & free pages, pages from the lower priority list will be "repurposed" and made available for the memory request.

Free & Zero list.  These are memory blocks that do not have recoverable memory and will be used for any application or  kernel component requesting memory.

The bottom graph shows a historical view of cache activity (file system requests for files that  might be  cached),  and the  effectiveness of the cache in having what is needed on-hand.   A miss occurs only if the requested file portion  is not in the  working set or may be recovered from one of the standby pools.  These values are as reported by the OS.

The text on the left of the display provides greater detail and additional interesting things to look at.

Click on the "Show Standby Repurposed" button to see information about standby list repurposing instead of cache effectiveness.

What does ATM cost?

TMurgent believes in giving back to the community.   ATM is free for for use by consultants and enterprises. The exception to this is that we ask ISVs that want to use this tool in their labs to contact us for a license.

How does this tool work?

The tool makes use of published APIs, when applicable, but also uses some unpublished kernel interfaces to obtain information or request system modifications. The tool understands the differences between versions of the operating system and adjusts to those differences.

What OSs will it run on?

The tool will run on Windows 2000 through Vista and Windows 7. At the time of this writing, the Windows 7 RTM version is the lastest tested. Only the 32-bit version will work on 32-bit operating systems, the 64-bit versions is needed on an x64 OS.

What is SuperFetch on Vista/Windows 7?

SuperFetch is new in these OSs. Normally the File System Cache works with opened files only. Once the last user of a file closes its handle, the file is flushed from cache (and goes to the standby list).

In these new OSs, there is a Windows Service that watches file usage patterns and recognizing these, will pre-cache files (or portions of files) into the Standby Cache. Or more precisely, the standby cache has been broken into three parts and these pre-cached files are added to the Standby-Reserve cache.

One example of this is when I boot my PC and log-in. SuperFetch will recognize from past occurrences that I am likely to open Outlook and, when it sees idle time, start reading in Outlook and my prf. Now when I do start Outlook, Outlook starts faster than it would have because all that time-consuming IO is avoided by being pre-cached.

But doesn't SuperFetch use up all my available memory?

What is important is the zero-page and free memory, and the zero page threads of  the system idle process make sure there is a sufficient supply of that. In fact, when needed, memory will be taken from the Standby Reserve cache before taking from the other two standy lists.

Doesn't the prefetching itself slow down my system?

Microsoft says no. I say "not as much as you would think". The requests to pre-cache happen in low CPU priority threads. In addition, these new OSs support prioritized IO, and this pre-fetch is using low priority IO requests. You can also say that the memory used is low priority since it uses the standby reserve, which is a lower priority than standby-normal priority or standby--core from the viewpoint of freeing up memory for the zero page list.  Still, only if your system is truely idle for a while would it be transparent.  My Vista PC with 2GB of ram will typically have a standby reserve of 500MB to 1.3tmtnGB of stuff - and that is a lot of I/O when it first comes in.  Unless I went for coffee (which I do a lot) I will notice it somewhat.  But overall, I believe my system operates faster  because of it.   And I can always test that by simply turning off the service (but I don't because it is helping).

How do I install it?

No installer. Just download, unzip, read the readme, and copy the exe somewhere convenient.

Where Can I download it?

Here are the ZIP packages:    ATM 32-bit (175KB )   or   ATM 64-bit (224KB )