Useful Tools for File System Developers

In this post I'll go over some of the tools I use pretty frequently to investigate file system behavior and debug my filters. I've noticed that some of the tools I use are somewhat unknown to other developers and so I hope this might save someone some time.

• FileTest - I've already mentioned this tool many times in my previous posts. It's extremely useful to me for investigating file system behavior as well as trying to reproduce bugs in my filter. I really like the ability to generate a breakpoint right before a certain operation is issued, which is very helpful when debugging certain code paths where a regular breakpoint will just be too noisy. In those cases I enable the breakpoint in FileTest and then when I triggers I do a "bp /t @$thread " and the breakpoint will only trigger in the context of that thread. It's also very useful to see how some of the structures for information classes and such are set by the file system. Overall, a great tool.
• ProcMon - I'm sure everybody knows of ProcMon already. I use it a lot when I want to investigate the behavior of a certain application (how does an application write to a file or how does it use a log or some such). It's also the first thing I try when I get bug reports where "application X works without the filter but fails when the filter is present". For these debugging scenarios I also found it very useful for something else: finding the log file for the application. Most properly written applications have a mechanism to log errors and/or warnings but since there are so many ways to do that in Windows I found that ProcMon is quite useful to find the log file (if there is one). I'm sure everyone reading my blog is a power-user of ProcMon so I won't go into more detail about the use cases and such. However, there is one trick that I'd like to mention that makes is very useful for file system developers: changing the altitude of ProcMon's filter. For file system filter developers this is incredibly useful because it allows them to see what the IO their filter issues looks like. Also, for debugging interop issues it's very useful to be able to put ProcMon between the two instances and see what's going on. First I capture ProcMon's configuration like so (make sure to capture it again when upgrading ProcMon):
  1. Start procmon.
  2. Open the registry entry for the filter: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\PROCMON20 (for the current version on one of my test machines)
  3. Export the key into a human-readable format (I use *.REG files).
  At this point I have a *.REG file with procmon's configuration which I can use whenever I need to load ProcMon at a different altitude. Please note that I remove the Enum key so what I have looks something like this :
  

  Windows Registry Editor Version 5.00
  
  [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\PROCMON20]
  
  [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\PROCMON20\Instances]
  "DefaultInstance"="Process Monitor Instance"
  
  [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\PROCMON20\Instances\Process Monitor Instance]
  "Altitude"="385200"
  "Flags"=dword:00000000
  

  So now whenever I need to have ProcMon filter at a different altitude I open the file and change the Altitude to what I need it to be. After that the steps are:
  
  1. Load the REG file in the registry
  2. Open regedit and go to the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\PROCMON20\Instances\Process Monitor Instance key
  3. Add DENY for Everyone for Delete and Set Value.
  4. start ProcMon
  5. use fltmc to make sure that procmon is loaded at the right altitude.
• fsutil - I don't know exactly why but this very useful tool is not as well known as I would have expected. It is a Microsoft tool that allows command line access to many file system functions that are useful for file system developers and which would otherwise require writing an in-house tool to access. This is a list of the ones I've used over the years but there are many more:
  • get file name by ID
  • set file shortname
  • query the allocated ranges for a file
  • query all sorts of information about the file system on a volume, including file system capabilities, configuration options, statistics and so on.
  • create and list hardlinks for a file
  • display information for reparse points (symlinks, mountpoints, directory junctions and custom ones)
  • work with sparse files (query and set the ranges, set the attribute and so on)
  • manage the USN journal (create, delete, query the entries for a file and so on)
• mountvol - this is another Microsoft tool. I found it very useful for volume management. It's great to create mount points and change (or remove) drive letters, get familiar with using the NT names for volumes and so on.
• diskpart - another Microsoft tool useful for creating VHDs and formatting volumes with various unusual characteristics. Useful for file system developers that want to test their filters with various file systems (see my previous post on Testing a Minifilter on More Filesystems: UDF and ExFAT).

These are the ones I use the most these days. I'll update the list if I remember another tool. Also please feel free to mention your favorite tool in the comments.