FltCreateSectionForDataScan

Thanks everyone for your suggestions. I really appreciate you took the time and sent me emails. Some of the topics you've suggested I won't be able to address for various reasons (mostly having to do with the legal implications of my doing that). However, I have a couple of topics that I might go over in the next couple of months. Incidentally, I also plan to change the frequency of my posts because of changes around my work so I'll probably update the blog whenever I get around to it. I still plan to answer questions though so feel free to leave comments and suggest topics (preferably in email). And now let's proceed to the topic at hand.

Someone suggested it would be useful to take a closer look at what FltCreateSectionForDataScan does and discuss how the same functionality could be achieved in previous OS releases. The FltCreateSectionForDataScan function is documented as being supported only in Win8+ but looking at the documentation page for the FltRegisterForDataScan function there is a mention about the FsRtlCreateSectionForDataScan function, which is documented as being available since Win 2000 SP4 and XPSP2, basically from the time FltMgr is available. A quick look in the debugger at FltCreateSectionForDataScan shows that it's really a wrapper around FsRtlCreateSectionForDataScan so I thought it might be useful to take a closer look at that:

1. As one might expect the FsRtlCreateSectionForDataScan function begins with various parameter checks, primarily to filter out various invalid parameters. There is also a check to see if the FileObject supports sections.
2. The next step is to set the TopLevelIrp to the value 1, which maps to FSRTL_FSP_TOP_LEVEL_IRP.
3. Once that is done there is a call to nt!FsRtlAcquireFileExclusiveCommon. This function is not documented anywhere and I could only find some references in crash dumps. However, looking for just FsRtlAcquireFileExclusive returns more hits and we can see that FsRtlAcquireFileExclusive doesn't do much beyond calling nt!FsRtlAcquireFileExclusiveCommon. In the ntifs.h file we see a comment to the effect that FsRtlAcquireFileExclusive is called from NtCreateSection to avoid deadlocks with file systems, so we can guess that this the same thing going on here. We can also guess that whatever FsRtlCreateSectionForDataScan does is probably pretty similar to NtCreateSection.
4. The next step is to get the file size. This is done by a call to nt!FsRtlGetFileSize, and the documentation for FsRtlGetFileSize states that "If you already own file system resources, you should call FsRtlGetFileSize instead of ZwQueryInformationFile, because attempting to acquire the file object lock would violate locking order and lead to deadlocks. The ZwQueryInformationFile function should be only if you do not already own file system resources.". This makes sense since the function has already acquired the file object lock in the previous step.
5. There is a quick check to see if the file is empty and the function should return with STATUS_END_OF_FILE.
6. Then, once everything is set up we have a call to nt!MmCreateSection. Calling this function directly is not supported by Microsoft and there is no documentation for how to do that. This is basically why FsRtlCreateSectionForDataScan was added. This can be inferred from the msdn page "Installing the Filter Manager rollup package for Windows XP SP2". If the function returns STATUS_FILE_LOCK_CONFLICT the call to nt!MmCreateSection is retried after delaying the thread by nt!FsRtlHalfSecond (I had no idea this existed, but I plan to use it from now on :)).
7. Once the section is created there is a call to nt!CcZeroEndOfLastPage. I've not been able to find any documentation for this function (though it's pretty obvious what it does) except in the NT Insider article "Cache Me if You Can: Using the NT Cache Manager".
8. Once this is done the function releases the locks so the rest of the function is done outside the critical region (that was set up around step 2).
9. The only interesting thing happening in the rest of the function is the creation of the handle for the section in the current process handle table. The rest of it just deals with cleanup and the return path from the function.

One thing to note is that FsRtlCreateSectionForDataScan just creates the section. There are A LOT of FltMgr functions related to these data sections and based on the names most of them are dealing with conflict resolution. I'm not sure what that's about yet and I haven't actually used them anywhere but if you plan to use this API directly in downlevel OSes then you should probably expect some sort of conflicts. Good luck!