The Flags of FILE_OBJECTs - Part I

In this set of posts I'd like to discuss the various flags defined for FILE_OBJECTs and how they are used in the system. The reason I'm doing this is because for most of them the documentation is scarce or wrong and I've been meaning to document them for a while. Please note that most of this information is based on my experience with them and is not meant to be an exhaustive list of how they might be used.

The only documentation I've found on MSDN is on the page about the FILE_OBJECT structure, but as I said before it's very scarce on the details. So without further delay let's start with the flags:

• FO_FILE_OPEN - 0x00000001 - this flag is documented as being deprecated. The sample file systems (FastFat, CDFS) don't even mention it anywhere. I've also been unable to find it set anywhere (I expected either the file system to set it or the IO manager to set it but as far as I can tell it's not really used at all).
• FO_SYNCHRONOUS_IO - 0x00000002 - this is a pretty complicated flag (in that it controls different types of behavior) . It can only be set during the create operation (before the FILE_OBJECT is sent through the IO stack) if the caller uses the FILE_SYNCHRONOUS_IO_ALERT or FILE_SYNCHRONOUS_IO_NONALERT options. If none of these is set then the FILE_OBJECT won't have the flag set. This flag controls a couple of different behaviors:
  • if the flag is set then the FILE_OBJECT->CurrentByteOffset field is updated after each IO operation to reflect the current file position pointer and also it enables using the FILE_OBJECT->CurrentByteOffset as the current file offset to perform IO at (FILE_USE_FILE_POINTER_POSITION can only be used for a file that has FO_SYNCHRONOUS_IO set). So in a way this flag controls whether the current position can be used for this FILE_OBJECT.
  • if the flag is set then the IO manager will try to synchronize operations using that FILE_OBJECT. So if multiple threads use the same FILE_OBJECT then the IO manager will only allow one operation at a time to proceed. Please note that this doesn't mean that file system filter will only see one operation at a time since it's possible that other components on the system issue IO directly on the FILE_OBJECT without synchronizing with the IO manager (pretty typical with the Cache Manager).
  • If the flag is set then IoIsOperationSynchronous (and its friend FltIsOperationSynchronous) will return TRUE (as long as this isn't an asynchronous paging IO request, in which case it doesn't matter whether FO_SYNCHRONOUS_IO is set). The return value of IoIsOperationSynchronous and FltIsOperationSynchronous might have deeper impact on how the request is processed by the file system or file system filters (which might decide to process it inline rather than queue it if it's synchronous).
• FO_ALERTABLE_IO - 0x00000004 - as the FILE_OBJECT structure documentation page mentions, this flag controls how the IO manager waits when using the FILE_OBJECT. If the flag is set then most waits (which can be waits to acquire the FILE_OBJECT to synchronize IO on it or just waiting for IO to complete) are alertable. The flag will be set depending on the presence of FILE_SYNCHRONOUS_IO_ALERT or FILE_SYNCHRONOUS_IO_NONALERT. The flag is only meaningful for FILE_OBJECTs for which FO_SYNCHRONOUS_IO is set and can be set and cleared in the Create path and using the FileModeInformation information class. Please note that setting or clearing it using FileModeInformation only works if the FILE_OBJECT has the FO_SYNCHRONOUS_IO set. Another thing to note is that file systems seem to ignore this flag.
• FO_NO_INTERMEDIATE_BUFFERING - 0x00000008 - this flag indicates whether the file was opened for non-cached IO. It maps to the FILE_NO_INTERMEDIATE_BUFFERING flag and can only be set during Create. It controls two different behaviors as well:
  • If it is set then there is no file system caching for this FILE_OBJECT. The implications here are pretty heavy. A lot of the cache management specific operations and flags are disabled (I'll discuss this more further down when I talk about some of the other flags). Also the file system code paths are quite different when dealing with cached IO and non-cached IO. This also means that for IO happening on this FILE_OBJECT the file system might need to perform flushes to keep the cached and non-cached views of the data coherent.
  • Another implication of this flag being set is that IO must be aligned to the sector size of the underlying device. This impacts reads and writes as well as setting the current file pointer for example. Unaligned operations on this FILE_OBJECT will generally fail at the IO manager level with STATUS_INVALID_PARAMETER.
• FO_WRITE_THROUGH - 0x00000010 - this flag indicates that any data written on this FILE_OBJECT should be flushed immediately to disk. It maps to the FILE_WRITE_THROUGH flag. It can be set in the Create file path and also at any time with the FileModeInformation information class. This flag cannot be set when FO_NO_INTERMEDIATE_BUFFERING is set (it makes no sense if there is no cache at all). Please note that this flag doesn't require that operations be aligned to the underlying sector size, since caching is still in effect, it's just that writes are immediately flushed to disk.

I'll stop here for today but I plan to address all the flags in the following posts.