More on Asynchronous Handles

Since we're on the subject of asynchronous handles I'd like to spend some time do discuss the create options that deal with synchronization and what their effects are. The documentation for ZwCreateFile() mentions three of them: SYNCHRONIZE, FILE_SYNCHRONOUS_IO_ALERT and FILE_SYNCHRONOUS_IO_NONALERT. The documentation describes briefly how they might be used but doesn't go into enough detail (in my opinion). So let's talk about each of the options.

• SYNCHRONIZE - this option is not specific to files. As you probably already know, the NT kernel makes use of objects that have a common subset of features. For example most objects can be used for signaling and thus can be waited on. Each object is signaled under specific circumstances (threads are signaled when they are terminated, files are signaled when IO has completed and so on). If an object is to be used for synchronization then the user must have SYNCHRONIZE access to it (the user generally has a handle to an object and so the handle must allow for SYNCHRONIZE access). In other words, if you open a handle to an object and don't specify SYNCHRONIZE, you can't wait on it. As I mentioned before, a FILE_OBJECT might be is signaled when an IRP has completed. The FILE_OBJECT will be signaled during IRP completion in most cases (there are some exceptions here) but only handles that were opened with SYNCHRONIZE can wait on it. Just specifying this for ZwCreateFile doesn't make the handle synchronous, but does allow a caller to issue requests that might be completed asynchronously (the caller gets STATUS_PENDING) and then to wait on the handle to know when the IO has completed. Of course, any IRP will signal the FILE_OBJECT so in a heavily asynchronous environment waiting on the handle is not particularly useful and an event should be used instead (the IO manager will signal the event instead of the FILE_OBJECT when the caller provides one). The MSDN page on Synchronous and Asynchronous I/O is an interesting read as well.
• FILE_SYNCHRONOUS_IO_ALERT - this means that he handle is synchronous. There is at most one request issued by the IO manager on that handle at any given time. Of course, other components in the system that roll their own IRPs can issue additional IRPs on the same FILE_OBJECT even if it's synchronous and if they don't use an event when doing so IRP completion might signal the FILE_OBJECT (I would call this a bug in the driver that does this). So calling a handle "synchronous" only means it's synchronous at a top level (Nt API level). When the user performs any operation on the user handle, the function won't return until the request is complete. It is therefore impossible for a user to queue more than one request on a handle using only one thread anyway. However, if a handle is synchronous and a user queues multiple requests on a synchronous handle by using multiple threads then the IO manager will wait for each request to complete before sending the next one down. In case of the FILE_SYNCHRONOUS_IO_ALERT flag specifically, all the waits will be alertable (if any APCs are queued they can be processed; a common case for this is when the thread is killed and the wait is abandoned).
• FILE_SYNCHRONOUS_IO_NON_ALERT - this is the same as above (the handle is synchronous and so on) but the difference from FILE_SYNCHRONOUS_IO_ALERT is that waits are not alertable, i.e. if any waits happen along the way they won't return until the objects they're waiting for are signaled (and APCs queued to such threads won't be processed).

Now, in terms of the FILE_OBJECT associated with the create, the SYNCHRONIZE option has no effect (the object itself can always be signaled, it's just that some handles might not be able to wait on it). However, if either the FILE_SYNCHRONOUS_IO_ALERT or the FILE_SYNCHRONOUS_IO_NON_ALERT flags are set then the FILE_OBJECT will have a special flag set, the FO_SYNCHRONOUS_IO. If the FILE_SYNCHRONOUS_IO_ALERT is specified then the FILE_OBJECT will also have the FO_ALERTABLE_IO (obviously if FILE_SYNCHRONOUS_IO_NON_ALERT is specified then only FO_SYNCHRONOUS_IO is set). Also, it's not possible to call create with FILE_SYNCHRONOUS_IO_ALERT or FILE_SYNCHRONOUS_IO_NON_ALERT without specifying SYNCHRONIZE, since it would be impossible for the IO manager to synchronize with the FILE_OBJECT.

When a handle is opened for asynchronous IO (FO_SYNCHRONOUS_IO is not set) and an Nt API is called that is synchronous (like NtQueryInformationFile() from my previous post) then the function will allocate an EVENT and pass it in the IRP to be signaled when the request has completed. If the FILE_OBJECT does have FO_SYNCHRONOUS_IO set then it will simply wait on the file handle.

One more thing to note is that it is possible for the synchronous mode to change during the lifetime of a FILE_OBJECT. Calling NtSetInformationFile() with the FileModeInformation information class allows the caller to change this parameter as well so filters must be careful not to assume that once the create is done that this can't change anymore (for example by caching this information or the result of some decision based on this information in a StreamHandleContext).

So now I'd like to talk a bit about what this might mean for filters and what sort of things a filter must be aware:

• Filters can detect whether a certain operation is synchronous by calling IoIsOperationSynchronous() or the FltMgr counterpart, FltIsOperationSynchronous(). Please note the documentation passage that states: "If the operation is an asynchronous paging I/O operation, the operation is asynchronous, even if one of the other conditions in this list is true". This means that if a file is used for asynchronous paging IO, whether the file has the FO_SYNCHRONOUS_IO flag is irrelevant, the request is still asynchronous. This information can be used as a hint about whether the filter can afford to hold the caller's IRP and perform whatever operations it requires inline or it should post the request to a worker queue or a different thread and return STATUS_PENDING to the caller. If the request is synchronous then it generally is easier to do things inline.
• If a filter is performing operations in response to a user request (where the filter stops the request for some time and does some work before releasing the operation) they must try to follow the user's instructions and if any wait is required it should be of the right type (if the user wants alertable waits then the filter should try to accommodate them).
• If a filter wants to use a FILE_OBJECT that it didn't open itself to perform IO then it must take great care to not do something that will signal the file handle. For minifilters this is fairly easy since FltMgr provides a large list of FltXxx APIs and the FltPerformAsynchronousIo() and FltPerformSynchronousIo() functions that help with that and take care of the complexity of issuing IO on a FILE_OBJECT that they don't own.
• Of course any minifilter that holds on to a user's request for a while should use a cancel-safe queue irrespective of whether the operation is synchronous or not. Using a cancel-safe queue is shown in the cancelSafe minifilter sample in the WDK and legacy filters have a similar API set (IoCsqXxx..) and if I remember correctly there was a sample for that as well.