The Flags of FILE_OBJECTs - Part II

In the post last week we discussed some of the various flags of the FILE_OBJECT, but since there are so many of them I left some for this week. So here we go again:

• FO_SEQUENTIAL_ONLY - 0x00000020 - this flag is set to indicate that the caller guarantees that this FILE_OBJECT will only be used for sequential requests (no seeks). It maps to the FILE_SEQUENTIAL_ONLY flag and is usually set when Creating the handle, though it's possible to set and query it through the FileModeInformation information class. This flag is simply an indication to the cache manager that it can be more aggressive about removing things from the file system cache (since the flag implies that once data at a certain offset was touched it will likely not be needed again). If the user sets this flag but doesn't honor the contract (i.e. if IO isn't actually sequential (so it's at random offsets in the file)) then the cache performance will not be optimal. This flag (FILE_SEQUENTIAL_ONLY) is useful when copying a file or scanning a file for hashing and so on. Without the FILE_SEQUENTIAL_ONLY flag the file will be kept in the cache and will compete with everything else in there, so it might evict useful data from the cache (for example you have an application open and you switch to explorer and copy some files around... if the files that are copied are opened with the FILE_SEQUENTIAL_ONLY flag then the cache will likely prioritize existing things in cache over the file contents so when you switch back to your application it will still be in the cache. Without the flag it is more likely that the application will be removed from the cache because of the file copy...).
• FO_CACHE_SUPPORTED - 0x00000040 - according to the MSDN page on the FILE_OBJECT structure this flag should only be set if the file system implements supports for the FSRTL_ADVANCED_FCB_HEADER structure. However, all windows file systems should implement this structure anyway. This flag is set by the filesystem as a means to keep track whether the open for this FILE_OBJECT had the FILE_NO_INTERMEDIATE_BUFFERING set (if FILE_NO_INTERMEDIATE_BUFFERING was set then this must be clear). This flag isn't terribly useful since the flag that controls most of the caching behavior is FO_NO_INTERMEDIATE_BUFFERING. I'm not sure why someone felt there was a need to add this flag when FO_NO_INTERMEDIATE_BUFFERING conveys the same information.
• FO_NAMED_PIPE - 0x00000080 - this flag indicates that the FILE_OBJECT is for a named pipe. This is set by the file system (NPFS in this case) so it's not available in preCreate. According to a post on NTFSD, this flag can also be set for the connection to the IPC$ by the redirector, even though that FILE_OBJECT is not backed by NPFS on the remote server. On the local system everything works pretty much as you expect, with the flag being set for all NPFS FILE_OBJECTs.
• FO_STREAM_FILE - 0x00000100 - this flag is set for FILE_OBJECTs created using the IoCreateStreamFileObject API (or friends: IoCreateStreamFileObjectEx, IoCreateStreamFileObjectEx2 and IoCreateStreamFileObjectLite). I've discussed stream FILE_OBJECTs in my post About IRP_MJ_CREATE and minifilter design considerations - Part V. Please note that this does NOT indicate the FILE_OBJECT is for an alternate data stream (I've seen some people confuse them). The two concepts (stream file objects and alternate data streams) are unrelated.
• FO_MAILSLOT - 0x00000200 - this flag is similar in intent with the FO_NAMED_PIPE flag, except that in this case it indicates that the underlying file system is the mailslots file system (MSFS). According to the OSR thread that I pointed to earlier, apparently for FILE_OBJECTs for remote file systems the redirector doesn't set the FO_MAILSLOT flag even if the remote FILE_OBJECT has it. But anyway for local FILE_OBJECTs this flag should be set just like the FO_NAMED_PIPE flag.
• FO_GENERATE_AUDIT_ON_CLOSE - 0x00000400 - as the MSDN page on the FILE_OBJECT structure points out this flag is deprecated. It has the same value as the next flag, FO_QUEUE_IRP_TO_THREAD.
• FO_QUEUE_IRP_TO_THREAD - 0x00000400 - the only information I've been able to find has also been the MSDN page on the FILE_OBJECT structure. According to that page it means that IRPs will not be queued to this FILE_OBJECT and will be queued to the thread instead (at least that's what I get from the name). This has to do with the thread agnostic IO that was a new feature for Vista. The idea is that instead of queuing IO to a thread there are cases where it can be queued to a FILE_OBJECT (as an optimization). This is described in more detail in a post by PaulSli on Doron's blog, New for Windows Vista: thread agnostic I/O. My guess is that this flag disables that optimization. I've seen this flag set occasionally but I've never had to figure out where it's set and under what circumstances that happens. Still, I hope that this bit of investigation helps.

This is it for today, these things take quite a while to investigate so I can't do too many of them at once.