Detaching Instances and FLTFL_POST_OPERATION_DRAINING

I've recently realized that FLTFL_POST_OPERATION_DRAINING is not a very well understood feature. This is even though it is pretty well documented on the page for the PFLT_POST_OPERATION_CALLBACK function. So I think that in addition to the documentation page it might help to take a look at what happens under the hood.

So once a minifilter instance attaches to a volume it will start receiving IRPs and the filter will process them. There are many cases where filters need to process the results of the operation after the file system has completed it so the filter would register PostOperation callbacks and the filter would return an appropriate status to indicate that it wants a that PostOp callback called for each specific operation. These IRPs are generally called in-flight requests and in many cases the filter has data allocated associated with the operation (buffers allocated and possibly even replaced on the request, contexts that are sent from the PreOp callback to the PostOp callback, references (if the filter references a context in PreOp and dereferences it in PostOp) and so on). In case the instance needs to be detached (the minifilter is unloaded or just one instance is detached from the underlying volume) from a live volume, IO will continue to flow on the volume. One way to detach is to stop receiving new IO requests and then just wait for all the in-flight operations to be completed by the file system. Once all the in-flight operations have completed then FltMgr can pretty much free the instance because there can not be any more IO on it.

Unfortunately, this approach has a pretty big disadvantage. It is impossible to predict when the file system will complete all in-flight IRPs. There are certain operations that are very long-lived (for example, the directory change notification IRPs can be held in the file system indefinitely) which would make the time it takes to unload a filter or to detach an instance completely unpredictable. FltMgr howver is smarter than that and what it actually does is that it always tracks for each instance which operations are in-flight at any given time. So when an instance needs to be detached or a filter unloaded, FltMgr can actually know that it doesn't need to call any PostOp callbacks for that filter or that instance. However, there is still data associated with the request, buffers that have been replaced, contexts that are allocated and so on. If FltMgr simply stops calling the PostOp callbacks for all in-flight operations then the allocated data and the buffers would never be freed and the references would be leaked.

This is where FLTFL_POST_OPERATION_DRAINING comes in. When an instance is detached, FltMgr disconnects the instances from the IO path and then calls each PostOp callback for all the in-flight operations with the FLTFL_POST_OPERATION_DRAINING flag set. When a PostOp callback is called with FLTFL_POST_OPERATION_DRAINING flag the callback must not do any actual work (like checking the status of the operation or touching the FLT_CALLBACK_DATA structure at all) and instead the filter must just release any buffers or references they might have allocated. For most filters this is generally similar to what they do when the operation is unsuccessful (though there might be cases where this is different so this is more of a hint than an actual guideline to implement this feature). It is important to note that when the PostOp callback is called with the FLTFL_POST_OPERATION_DRAINING flag the actual operation might have already been completed or it might still be in the file system somewhere so the filter must not make any assumptions about the status of the actual operation.