Names in Minifilters - Implementing Name Provider Callbacks

Since we're going to talk a bit more about names, one important aspect to cover is how to implement minifilters that change the namespace in some way. Of course, this is a very large topic so in this post I'm going to cover one particular aspect of that, how to implement the name provider callbacks. The "name provider" callbacks are in fact two members of the FLT_REGISTRATION structure, GenerateFileNameCallback and NormalizeNameComponentCallback (generally referred to as the generate callback and the normalize callback). There are two other callbacks related to this (NormalizeContextCleanupCallback and NormalizeNameComponentExCallback) but they should be pretty easy to figure out. Strangely enough Microsoft doesn't currently provide any sample on how these callbacks are supposed to look like even in a basic case so I'll do that in this post. Please note that this is written almost from scratch. It is very loosely based on some existing code I have but that was way too complicated for the purpose of this post. I just wanted to show what the callbacks are supposed to do in principle. So the code here hasn't been extensively tested, it might fail in unusual circumstances and so on. It should only used as a reference and not in a production environment (not that it does all that much anyway). With that said, if you do run this code and it fails under some circumstances or you spot an error just by looking at it, please let me know and I'll investigate and update it so that everyone benefits from it.

I'll be referring to minifilters that implement these name provider callbacks as name provider minifilters, or simply name providers.

First let's talk a bit about why these callbacks exist. As you may have read on this blog or you may know from experience, name generation is a pretty complicated business (I use name generation in the general sense, referring both to creating a name for a file and to normalizing that name). In preCreate it might require looking in the FileObject and RelatedFileObject, or it might require looking up a fileID into a table to get the name and so on. If there is support for links (hardlinks for example) then the name depends on the FILE_OBJECT that's asking for it. For filters, it also matters if the file object is going to be a target of some operation that changes the namespace (rename, hardlink), in which case the name during the preOp is different from the name in postOp. Then you have the tunnel cache which might change the name and so on. Legacy filters spent a lot of time (both during development and at runtime) trying to get names for files, and so the Filter Manager team at MS started the project with the intention to simplify things. In the process of doing that they wrote a lot of code (I've heard that about 20% of the initial FltMgr code was dedicated to this) and they've noticed a couple of problems. First, performance isn't all that great and second (and this is the real nasty discovery in my opinion), that there is still a need to violate the strict layering rules FltMgr tries hard to obey (I haven't given it enough thought to see if there may have been another way so I'll just take their word for it). So they tried to address both these issues by creating a model where only minifilters that actually need to be involved in the name generation path (namespace virtualization and such) need to see the ugliness, while the rest of them can just enjoy the rather simple abstraction of calling FltGetFileNameInformation and getting back a name that is ready to be (mis)used.

For name provider minifilters, FltMgr requires that they implement two callbacks that it will call when generating (or normalizing) a name. Moreover, the IO generated by FltMgr while trying to generate these names will only be shown to these types of minifilters (name providers). This is a pretty serious decision because the minifilter model is designed so that minifilters really do see all IO happening on a volume, so excluding a certain class of IO was not a decision to be taken lightly. However, the performance benefits were enough that it seemed justified and in terms of what IO is shown to minifilters, pretty much all minifilters that I know of work just fine without processing that FltMgr IO (in fact, it comes as a surprise to most people that there is some IO happening behind the scenes which shows that it is truly transparent). Another problem with that IO is that, as I said before, it might violate layering. Specifically, the callbacks might be called when minifilters below them call FltGetFileNameInformation, which means that the developers of these kinds of minifilters need to be extremely careful when implementing their filters. In conclusion, my advice is that unless a minifilter actually MUST implement a name provider it should really avoid doing so, no matter how much they want to process "all IO". In general a minfilter MUST implement a name provider if it implements some sort of name virtualization scheme where the name of a file below their layer is different than the one below or where they take over part of the namespace or if the verifier warning I was talking about in this post occasionally pops up.

Let's not waste more time on warnings and talk about how the callbacks need to be implemented and what they should do.

GenerateFileNameCallback

The generate callback should be thought of as "the function that gets called when someone asks for an opened name" (FLT_FILE_NAME_OPENED). Its purpose is to return a file name given a FILE_OBJECT and a FLT_CALLBACK_DATA structure. There are a couple of important points about this function:

• The FLT_CALLBACK_DATA structure might be missing if the request for the name does not come from a component that is involved in an IO operation (for example, if a minifilter is also registered to receive process creation notifications and it wants to call FltGetFileNameInformation from that callback it can't because it doesn't have a FLT_CALLBACK_DATA structure; in that case its only recourse is to call FltGetFileNameInformationUnsafe). However, the minifilter writer can safely assume that either the FILE_OBJECT is opened (FILE_OBJECT->FsContext is not NULL) or there is a FLT_CALLBACK_DATA structure. It is impossible to get an unopened FILE_OBJECT outside of the IRP_MJ_CREATE path and in that path a minifilter must never call FltGetFileNameInformationUnsafe anyway. This might not strike you as very important but it matters a lot for case sensitivity (that's what you get for changing names...). The information about whether operations on a file are to be case sensitive or not is stored in a file when the file is opened (FO_OPENED_CASE_SENSITIVE). However, if the file is not yet open then the information can be found in the IRP_MJ_CREATE request, in the IO_STACK_LOCATION->Flags for legacy filters and FLT_CALLBACK_DATA->Iopb->OperationFlags for minifilters (SL_CASE_SENSITIVE).
  
• The filter has the option to tell FltMgr whether the name it returns should be cached or not. This is very important because if FltMgr caches the wrong name it will run into very interesting issues that are pretty complicated to debug. I've had a lot of fun that way. On the other hand, if the minifilter always tells FltMgr not to cache the name then performance will suffer greatly. When to cache really depends on minifilter architecture so there isn't much more general advice I can give.
  
• In general a minifilter should never return a name it is not prepared to handle in the future. For example, I've seen cases where a minifilter (A) was doing something like returning a name that was only valid during preCreate (a GUID that the minifilter used as a key in a hash and that was used to get a real file name below the minifilter's layer, after which the GUID was discarded). The minifilter then got in trouble when some other minifilter (B) used that name later on to open their own handle to the same file and minifilter A no longer had the GUID in its internal hash and so it had no idea what the real file was. I guess my advice is to try no to do anything too fancy here.
  

NormalizeNameComponentCallback

The normalize callback is called when someone asks for a normalized name. FltMgr gets a regular name (using the generate callback) and then it looks at each component and if it thinks it might be a short name it calls the name providers with the parent directory path and the name of the component it is trying to normalize. Here are some of the important things to mention:

• This function has a lot of potential for recursion. Try calling FltGetFileNameInformation to get a normalized name in preCreate from your own name provider and you'll see what I'm talking about. Don't use a lot of stack and try to avoid recursion as much as possible. This will require quite a lot of ingenuity to work around.
  
• The name of a file can be different inside a transaction and so any IO you perform must be in the context of that transaction. Possibly other operations (registry lookups ?) will need to be transacted as well.
  

And finally, here is the code that can be plugged into the passthrough sample to make it filter name provider requests:

 BOOLEAN  
 PtDoRequestOperationStatus(  
   __in PFLT_CALLBACK_DATA Data  
   );  
 NTSTATUS PtNormalizeNameComponentExCallback(  
   __in   PFLT_INSTANCE Instance,  
   __in_opt PFILE_OBJECT FileObject,  
   __in   PCUNICODE_STRING ParentDirectory,  
   __in   USHORT VolumeNameLength,  
   __in   PCUNICODE_STRING Component,  
   __out  PFILE_NAMES_INFORMATION ExpandComponentName,  
   __in   ULONG ExpandComponentNameLength,  
   __in   FLT_NORMALIZE_NAME_FLAGS Flags,  
   __inout PVOID *NormalizationContext  
   );  
 NTSTATUS PtNormalizeNameComponentCallback(  
   __in   PFLT_INSTANCE Instance,  
   __in   PCUNICODE_STRING ParentDirectory,  
   __in   USHORT VolumeNameLength,  
   __in   PCUNICODE_STRING Component,  
   __out  PFILE_NAMES_INFORMATION ExpandComponentName,  
   __in   ULONG ExpandComponentNameLength,  
   __in   FLT_NORMALIZE_NAME_FLAGS Flags,  
   __inout PVOID *NormalizationContext  
   );  
 NTSTATUS PtGenerateFileNameCallback(  
   __in   PFLT_INSTANCE Instance,  
   __in   PFILE_OBJECT FileObject,  
   __in_opt PFLT_CALLBACK_DATA CallbackData,  
   __in   FLT_FILE_NAME_OPTIONS NameOptions,  
   __out   PBOOLEAN CacheFileNameInformation,  
   __out   PFLT_NAME_CONTROL FileName  
   );  
 //  
 // Assign text sections for each routine.  
 //  
 #ifdef ALLOC_PRAGMA  
 #pragma alloc_text(INIT, DriverEntry)  
 ...  
 //  
 // This defines what we want to filter with FltMgr  
 //  
 CONST FLT_REGISTRATION FilterRegistration = {  
   sizeof( FLT_REGISTRATION ),     // Size  
   FLT_REGISTRATION_VERSION,      // Version  
   0,                 // Flags  
   NULL,                // Context  
   Callbacks,             // Operation callbacks  
   PtUnload,              // MiniFilterUnload  
   PtInstanceSetup,          // InstanceSetup  
   PtInstanceQueryTeardown,      // InstanceQueryTeardown  
   PtInstanceTeardownStart,      // InstanceTeardownStart  
   PtInstanceTeardownComplete,     // InstanceTeardownComplete  
   PtGenerateFileNameCallback,     // GenerateFileName  
   PtNormalizeNameComponentCallback,  // NormalizeNameComponent  
   NULL,                // NormalizeContextCleanup  
 #if FLT_MGR_LONGHORN  
   NULL,                // TransactionNotification  
   PtNormalizeNameComponentExCallback, // NormalizeNameComponentEx  
 #endif // FLT_MGR_LONGHORN  
 };  
 ...  
 NTSTATUS PtGenerateFileNameCallback(  
   __in   PFLT_INSTANCE Instance,  
   __in   PFILE_OBJECT FileObject,  
   __in_opt PFLT_CALLBACK_DATA CallbackData,  
   __in   FLT_FILE_NAME_OPTIONS NameOptions,  
   __out   PBOOLEAN CacheFileNameInformation,  
   __out   PFLT_NAME_CONTROL FileName  
   )  
 {  
   NTSTATUS status = STATUS_SUCCESS;  
   PFLT_FILE_NAME_INFORMATION belowFileName = NULL;  
   PT_DBG_PRINT( PTDBG_TRACE_ROUTINES,  
          ("PassThrough!PtGenerateFileNameCallback: Entered\n") );  
   __try {  

     //
     //  We expect to only get requests for opened and short names.
     //  If we get something else, fail. Please note that it is in
     //  fact possible that if we get a normalized name request the
     //  code would work because it's not really doing anything other 
     //  than calling FltGetFileNameInformation which would handle the
     //  normalized name request just fine. However, in a real name 
     //  provider this might require a different implementation. 
     //

     if (!FlagOn( NameOptions, FLT_FILE_NAME_OPENED ) && 
         !FlagOn( NameOptions, FLT_FILE_NAME_SHORT )) {

         ASSERT(!"we have a received a request for an unknown format. investigate!");

         return STATUS_NOT_SUPPORTED ;
     }

     //  
     // First we need to get the file name. We're going to call   
     // FltGetFileNameInformation below us to get the file name from FltMgr.   
     // However, it is possible that we're called by our own minifilter for   
     // the name so in order to avoid an infinite loop we must make sure to   
     // remove the flag that tells FltMgr to query this same minifilter.   
     //  
     ClearFlag( NameOptions, FLT_FILE_NAME_REQUEST_FROM_CURRENT_PROVIDER );  
     //  
     // this will be called for FltGetFileNameInformationUnsafe as well and  
     // in that case we don't have a CallbackData, which changes how we call   
     // into FltMgr.  
     //  
     if (CallbackData == NULL) {  
       //  
       // This must be a call from FltGetFileNameInformationUnsafe.  
       // However, in order to call FltGetFileNameInformationUnsafe the   
       // caller MUST have an open file (assert).  
       //  
       ASSERT( FileObject->FsContext != NULL );  
       status = FltGetFileNameInformationUnsafe( FileObject,  
                            Instance,  
                            NameOptions,  
                            &belowFileName );   
       if (!NT_SUCCESS(status)) {  
         __leave;  
       }                              
     } else {  
       //  
       // We have a callback data, we can just call FltMgr.  
       //  
       status = FltGetFileNameInformation( CallbackData,  
                         NameOptions,  
                         &belowFileName );   
       if (!NT_SUCCESS(status)) {  
         __leave;  
       }                              
     }  
     //  
     // At this point we have a name for the file (the opened name) that   
     // we'd like to return to the caller. We must make sure we have enough   
     // buffer to return the name or we must grow the buffer. This is easy   
     // when using the right FltMgr API.  
     //  
     status = FltCheckAndGrowNameControl( FileName, belowFileName->Name.Length );  
     if (!NT_SUCCESS(status)) {  
       __leave;  
     }  
     //  
     // There is enough buffer, copy the name from our local variable into  
     // the caller provided buffer.  
     //  
     RtlCopyUnicodeString( &FileName->Name, &belowFileName->Name );   
     //  
     // And finally tell the user they can cache this name.  
     //  
     *CacheFileNameInformation = TRUE;  
   } __finally {  
     if ( belowFileName != NULL) {  
       FltReleaseFileNameInformation( belowFileName );        
     }  
   }  
   return status;  
 }  
 NTSTATUS PtNormalizeNameComponentCallback(  
   __in   PFLT_INSTANCE Instance,  
   __in   PCUNICODE_STRING ParentDirectory,  
   __in   USHORT VolumeNameLength,  
   __in   PCUNICODE_STRING Component,  
   __out  PFILE_NAMES_INFORMATION ExpandComponentName,  
   __in   ULONG ExpandComponentNameLength,  
   __in   FLT_NORMALIZE_NAME_FLAGS Flags,  
   __inout PVOID *NormalizationContext  
   )  
 {  
   //  
   // This is just a thin wrapper over PtNormalizeNameComponentExCallback.   
   // Please note that we don't pass in a FILE_OBJECT because we don't   
   // have one.   
   //  
   return PtNormalizeNameComponentExCallback( Instance,  
                         NULL,  
                         ParentDirectory,   
                         VolumeNameLength,   
                         Component,   
                         ExpandComponentName,  
                         ExpandComponentNameLength,  
                         Flags,  
                         NormalizationContext );  
 }  
 NTSTATUS PtNormalizeNameComponentExCallback(  
   __in   PFLT_INSTANCE Instance,  
   __in_opt PFILE_OBJECT FileObject,  
   __in   PCUNICODE_STRING ParentDirectory,  
   __in   USHORT VolumeNameLength,  
   __in   PCUNICODE_STRING Component,  
   __out  PFILE_NAMES_INFORMATION ExpandComponentName,  
   __in   ULONG ExpandComponentNameLength,  
   __in   FLT_NORMALIZE_NAME_FLAGS Flags,  
   __inout PVOID *NormalizationContext  
   )  
 {  
   NTSTATUS status = STATUS_SUCCESS;  
   HANDLE parentDirHandle = NULL;  
   OBJECT_ATTRIBUTES parentDirAttributes;  
   BOOLEAN isDestinationFile;  
   BOOLEAN isCaseSensitive;  
   IO_STATUS_BLOCK ioStatus;  
 #if FLT_MGR_LONGHORN  
   IO_DRIVER_CREATE_CONTEXT driverContext;  
   PTXN_PARAMETER_BLOCK txnParameter = NULL;  
 #endif // FLT_MGR_LONGHORN  
   PT_DBG_PRINT( PTDBG_TRACE_ROUTINES,  
          ("PassThrough!PtNormalizeNameComponentExCallback: Entered\n") );  
   __try {  
     //  
     // Initialize the boolean variables. we only use the case sensitivity  
     // one but we initialize both just to point out that you can tell   
     // whether Component is a "destination" (target of a rename or hardlink  
     // creation operation).  
     //  
     isCaseSensitive = BooleanFlagOn( Flags,   
                      FLTFL_NORMALIZE_NAME_CASE_SENSITIVE );  
     isDestinationFile = BooleanFlagOn( Flags,   
                       FLTFL_NORMALIZE_NAME_DESTINATION_FILE_NAME );  
     //  
     // Open the parent directory for the component we're trying to   
     // normalize. It might need to be a case sensitive operation so we   
     // set that flag if necessary.  
     //  
     InitializeObjectAttributes( &parentDirAttributes,  
                   (PUNICODE_STRING)ParentDirectory,   
                   OBJ_KERNEL_HANDLE | (isCaseSensitive ? OBJ_CASE_INSENSITIVE : 0 ),  
                   NULL,  
                   NULL );  
 #if FLT_MGR_LONGHORN  
     //  
     // In Vista and newer this must be done in the context of the same  
     // transaction the FileObject belongs to.      
     //  
     IoInitializeDriverCreateContext( &driverContext );  
     txnParameter = IoGetTransactionParameterBlock( FileObject );  
     driverContext.TxnParameters = txnParameter;  
     status = FltCreateFileEx2( gFilterHandle,  
                   Instance,  
                   &parentDirHandle,  
                   NULL,  
                   FILE_LIST_DIRECTORY | SYNCHRONIZE,  
                   &parentDirAttributes,  
                   &ioStatus,  
                   0,  
                   FILE_ATTRIBUTE_NORMAL | FILE_ATTRIBUTE_DIRECTORY,   
                   FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,  
                   FILE_OPEN,  
                   FILE_DIRECTORY_FILE | FILE_SYNCHRONOUS_IO_NONALERT,  
                   NULL,  
                   0,  
                   IO_IGNORE_SHARE_ACCESS_CHECK,  
                   &driverContext );  
 #else // !FLT_MGR_LONGHORN  
     //  
     // preVista we don't care about transactions  
     //  
     status = FltCreateFile( gFilterHandle,  
                 Instance,  
                 &parentDirHandle,  
                 FILE_LIST_DIRECTORY | SYNCHRONIZE,  
                 &parentDirAttributes,  
                 &ioStatus,  
                 0,  
                 FILE_ATTRIBUTE_NORMAL | FILE_ATTRIBUTE_DIRECTORY,   
                 FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,  
                 FILE_OPEN,  
                 FILE_DIRECTORY_FILE | FILE_SYNCHRONOUS_IO_NONALERT,  
                 NULL,  
                 0,  
                 IO_IGNORE_SHARE_ACCESS_CHECK );  
 #endif // FLT_MGR_LONGHORN  
     if (!NT_SUCCESS(status)) {  
       __leave;  
     }  
     //  
     // Now that we have a handle to the parent directory of Component, we  
     // need to query its long name from the file system. We're going to use  
     // ZwQueryDirectoryFile because the handle we have for the directory   
     // was opened with FltCreateFile and so targeting should work just fine.  
     //  
     status = ZwQueryDirectoryFile( parentDirHandle,  
                     NULL,  
                     NULL,  
                     NULL,  
                     &ioStatus,  
                     ExpandComponentName,  
                     ExpandComponentNameLength,  
                     FileNamesInformation,  
                     TRUE,  
                     (PUNICODE_STRING)Component,  
                     TRUE );   
   } __finally {  
     if (parentDirHandle != NULL) {  
       FltClose( parentDirHandle );  
     }  
   }  
   return status;  
 }  

Update 04/12/2011: added check and assert to PtGenerateFileNameCallback.
Update 03/08/2012: see this post.