Thursday, April 28, 2011

When to use FltGetNewSystemBufferAddress

One of the really annoying operations that some file system filters need to deal with is the directory change notification. This comes in the form of an IRP_MJ_DIRECTORY_CONTROL with an IRP_MN_NOTIFY_CHANGE_DIRECTORY minor function. This operation allows a client of the file system (a user mode application for example) to be told when someone changes something about the contents of a directory. For more information on how this is exposed through the Win32 API, see the MSDN page on Obtaining Directory Change Notifications. In the file system this is implemented using a set of FsRtl routines (FsRtlNotifyFullChangeDirectory, FsRtlNotifyFullReportChange) and the FastFat source code in the WDK is a pretty good example on how a file systems uses these APIs.

Not all file system filters need to deal with this operation, the pleasure is typically reserved to namespace virtualization or content virtualization (which are filters that don't change the namespace but only what the contents of the objects in the namespace are, like encryption or compression filters) filters. What makes this complicated is that a filter might typically need to generate more events than the file system (if it manages virtual objects then it needs to send notifications for those) or to hide some events the file system generates (if the filter does some things "behind the scenes" which might trigger events that shouldn't been seen above the filter) or typically both. Windows itself uses these notifications (in explorer.exe) and so not handling them properly usually results in some pretty interesting behavior.

Now, in terms of implementation, the FsRtls have one particular interesting feature. Most notification mechanisms in the file systems are based on the same general pattern: whoever needs the notification sends an IRP into the FS, the FS pends the IRP and whenever it needs to signal that whatever thing the caller wanted to be notified about has happened it simply completes the IRP. This is the same story with oplocks and it is a pretty good mechanism in general. However, there is an interesting problem facing whoever is implementing this mechanism. What if the caller of this request needs some additional information about what has happened ? Well, they can just pass in a buffer in the IRP, right ? The problem here is that in order for the file system to be able to fill in that buffer it must have either a MDL for it or it must be in the right process context (also keep in mind that these requests are quite long-lived; they might be stuck in the file system for minutes or hours). However, MDLs and physical pages are precious resources (well, less so since having 4GB of RAM is pretty common these days but they used to be more precious back when this was designed) and being in the right process context is more complicated when returning STATUS_PENDING. So the FsRtl function in this case (where there isn't already a MDL and the buffer is not a SystemBuffer) will simply pend the request as is and when it needs to complete it will allocate a system buffer, put it into SystemBuffer and send it back up to the IO manager, which will know that it needs to use the data in SystemBuffer to copy it into the user's buffer and then free SystemBuffer (this is done in other places and as far as I can tell IRP_DEALLOCATE_BUFFER tells the IO manager to do this).

This is a particularly interesting problem for minifilters because as I've explained in the post about the COMPLETION_NODE, the parameters that are shown to the minifilter in the postOp callback are the same parameters that it has seen during the preOp callback. Those parameters didn't have anything in SystemBuffer, so how is the minifilter going to detect that this sort of thing has happened and how can it access that buffer ? This is where FltGetNewSystemBufferAddress() comes in handy. A minifilter can detect that this has happened by checking whether the FLTFL_CALLBACK_DATA_NEW_SYSTEM_BUFFER flag is set and if it is it can call the API to get a pointer to that buffer.

Let's take a look at this in the debugger (this is easy to repro, just set in the PassThrough sample a break in PtPostOperationPassThrough when FLTFL_CALLBACK_DATA_NEW_SYSTEM_BUFFER is set for an IRP_MJ_DIRECTORY_CONTROL function with an IRP_MN_NOTIFY_CHANGE_DIRECTORY minor function). Also remember that this is Win7 specific so you won't see your break trigger on anything before that :

1: kd> !fltkd.cbd @@(Data)

IRP_CTRL: 924482c8  DIRECTORY_CONTROL (12) [00180001] Irp PostOperation
Flags                    : [10000004] DontCopyParms FixedAlloc
Irp                      : 9244da98 
DeviceObject             : 92fa1c68 "\Device\HarddiskVolume2"
FileObject               : 92f45570 
CompletionNodeStack      : 92448380   Size=5  Next=1
SyncEvent                : (924482d8)
InitiatingInstance       : 00000000 
SwappedBufferMdl         : 00000000 
CallbackData             : (92448328)
 Flags                    : [00180001] Irp PostOperation +100000!!
 Thread                   : 943d5560 
 Iopb                     : 92448398 
 RequestorMode            : [01] UserMode
 IoStatus.Status          : 0x00000000 
 IoStatus.Information     : 00000012 
 TagData                  : 00000000 
 FilterContext[0]         : 00000000 
 FilterContext[1]         : 00000000 
 FilterContext[2]         : 00000000 
 FilterContext[3]         : 00000000 

   Cmd     IrpFl   OpFl  CmpFl  Instance FileObjt Completion-Context  Node Adr
--------- -------- ----- -----  -------- -------- ------------------  --------
 [0,0]    00000000  00   0000   00000000 00000000 00000000-00000000   924484a0
     Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000
 [0,0]    00000000  00   0000   00000000 00000000 00000000-00000000   92448458
     Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000
 [0,0]    00000000  00   0000   00000000 00000000 00000000-00000000   92448410
     Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000
 [12,2]   00060000  00   0000   930a8978 92f45570 9605c6c8-00000000   924483c8
            ("FileInfo","FileInfo")  fileinfo!FIPostOperationCommonCallback 
     Args: 00000800 00000015 00000000 00000000 087a7830 0000000000000000
>[12,2]   00060000  00   0000   923cab40 92f45570 a0f51140-00000000   92448380
            ("PassThrough","PassThrough Instance")  PassThrough!PtPostOperationPassThrough 
     Args: 00000800 00000015 00000000 00000000 087a7830 0000000000000000
Working IOPB:
[12,2]   00060000  00          930a8978 92f45570                     92448354
     Args: 00000800 00000015 00000000 00000000 087a7830 0000000000000000
1: kd> dt 9244da98 nt!_IRP AssociatedIrp.SystemBuffer
   +0x00c AssociatedIrp              : 
      +0x000 SystemBuffer               : 0xb1f87800 Void
1: kd> dt 92448380 fltmgr!_COMPLETION_NODE DataSnapshot.Parameters.DirectoryControl.NotifyDirectory.
   +0x018 DataSnapshot                                              : 
      +0x010 Parameters                                                : 
         +0x000 DirectoryControl                                          : 
            +0x000 NotifyDirectory                                           : 
               +0x000 Length                                                    : 0x800
               +0x004 CompletionFilter                                          : 0x15
               +0x008 Spare1                                                    : 0
               +0x00c Spare2                                                    : 0
               +0x010 DirectoryBuffer                                           : 0x087a7830 Void
               +0x014 MdlAddress                                                : (null) 
So as you can see by looking at this callback data, the DirectoryBuffer is a user mode address and there is no MDL so the minifilter has no way to get to the actual buffer being returned by the file system. Pre Win7 this would require that a minifilter that needs to look at the buffer in the post Op to allocate a MDL for the buffer in the preOp or to replace the buffer with their own buffer. Another thing to note is that the fltkd extension doesn't know about FLTFL_CALLBACK_DATA_NEW_SYSTEM_BUFFER and so it displays it by value (I've highlighted it).