More on Instances and Volumes

I've recently been playing some with instances and I've come across a couple of things that I wanted to share. Prerequisites for this discussion are my old posts on the FLT_INSTANCE structure and the FLT_VOLUME structure.

In Filter Manager terms, a volume is an attachment of fltmgr to the file system stack on a volume. The volume maps to a to a FltMgr DEVICE_OBJECT attached to a file system VDO. In most cases, where there are no legacy filters on a system, the volume represents the whole IO stack between the IO manager and the file system. However, when legacy filters are present on the stack multiple volumes can be attached on each file system stack. See this picture which I'm reusing from my FLT_VOLUMES post.

An interesting thing to note is the way FltMgr attaches to a file system stack. The simplified view is that FltMgr attaches a frame between each legacy filter, but that's not an accurate picture in a couple of ways. First, a legacy filter can attach only to some volumes, which means that on the other volumes there might be no legacy filter at all. Nevertheless, for consistency reasons, FltMgr attaches a DEVICE_OBJECT even if there are no DEVICE_OBJECTs belonging to other legacy filters on a volume. Also, since there is no mechanism to know when a device was attached to a device stack, FltMgr can't know when a legacy filters attaches to a certain device stack, which prevents it from being able to attach immediately on top of each legacy filter. So FltMgr only looks at the file system stack and tries to attach when a minifilter is loaded. At that time FltMgr tries to figure out which frame it should belong to, depending on the altitude (and in case you were wondering, the altitude comes from the default instance, which is why FltRegisterFilter() might fail with STATUS_OBJECT_NAME_NOT_FOUND if there is no default instance specified in the INF file). If no frame already exists where the minifilter altitude fits (and since Frame 0 starts at altitude 0 this scenario usually happens when the altitude of the new minifilter is higher than the altitude of the highest frame), then FltMgr looks at whether the top frame has any legacy filter attached on top. If not it will simply increase the highest altitude on that frame and loads the minifilter there. However, if a legacy filter has attached to the top frame then in Vista and newer OSes FltMgr tries to figure out what the altitude of that legacy filter is based on the Group (as in LoadOrderGroup) and then it grows the top of the highest frame (it increases the altitude) up to the altitude associated with that Group. Incidentally this is another good reason for legacy filters to use the appropriate Group. This way they can benefit to some extent from the layering guaranteed by FltMgr. Anyway, if the altitude is higher than the altitude of the top frame even after it was extended (again, this is only true for Vista and newer OSes, in XP the altitude on the frame is not increased) then a new frame is needed and so FltMgr proceeds to allocate a new frame and attach a new set of DEVICE_OBJECTs to each stack. This can have a couple of implications:

• There can be multiple legacy filters directly on top of each other, if no minifilter was loaded between the time when the first legacy filter was attached and the time when the second legacy filter was attached.
  
• There can be some volumes on which there are only FltMgr DEVICE_OBJECTs directly on top of each other. This should have no impact on minifilter developers but it might surprise someone looking a the stack in the debugger. This is actually quite common and it's perfectly fine.
  
• In extreme cases, it's possible that on one volume a legacy filter is attached above a certain frame while on a different volume it is attached below that frame. I've never seen this happen but I can imagine it would if the legacy filter attaches to volumes late (when some user mode apps requests attachment) or if the attachment happens to race with a minifilter loading.
  

An instance is an attachment of a filter to a certain file system volume. The notable thing about this is that an instance can be attached at multiple altitudes on the same volume. The altitudes at which an instance can attach are limited, however, by the altitude range of the frame. In other words, once a filter is loaded it is associated with a frame and it can only create instances at altitudes within that frame. Why would a filter create multiple instances on the same volume? One good reason for that is to test that the filter can attach above itself, which is a good way to test that the design is safe and it doesn't violate any layering rules. Another reason might be to analyze the behavior of a specific filter. In this case one might attach logging instances above and below it.

One decision a file system filter developer must make pretty early on is whether the filter should attach to volumes automatically or whether it needs manual attachment. For manual attachments a minifilter can use the FltAttachVolume() and the FltAttachVolumeAtAltitude() functions, but surprisingly these functions lack a context parameter. Looking at the PFLT_INSTANCE_SETUP_CALLBACK callback, we can see there is no callback parameter being passed as well (indicating that this is a design decision rather than a bug with the APIs). This can be problematic for filters that behave differently depending on which volume the filter is attached to. For example, imagine there is a filter that implements some form of file-level redundancy by duplicating some of the operations that happen for a file on a volume on a file another volume. This implies that when the filter starts working it might need to be attached to both volumes and it might need to know which is the target instance and which is the destination instance. One possible workaround would be to use an instance context for each instance that contains information about the role of the instance. This way a filter can call FltAttachVolume() or FltAttachVolumeAtAltitude() and if the call is successful it can use the pointer to the new instance to call FltSetInstanceContext() on that instance and inform the instance on the role it must perform. This is a rather unusual mechanism (passing the context to a callback is by far the more prevalent model in Windows) and the only reason I can think this was done this way is because of FilterAttach() and FilterAttachAtAltitude() for which passing in a context is not possible (passing in a pointer from kernel mode to user mode is not a good idea).

Finally, one last thing I'd like to point out is that there are two similar types of contexts, a volume context and an instance context. The vast majority of filters only have at most one instance per volume and so from a functional perspective they are equivalent. The instance context however is much faster to access because it is pretty much attached to the FLT_INSTANCE structure (so it's just a pointer deref) whereas the volume context is stored in some hash structure with the filter as the hash key so any lookup implies locking and walking the hash structure, which is much more costly.

So the couple of ideas that are important to remember from this post:

• All filters must have a default instance, otherwise FltRegisterFilter() will fail with STATUS_OBJECT_NAME_NOT_FOUND (which incidentally is not documented as a possible return value).
  
• When testing for interop with legacy filters, try to load the legacy both above your filter and below your filter (that might not be necessary on Vista+ environments where the legacy filter uses a Group, which might guarantee a fixed position relative to your filter).
  
• Use instance contexts always instead of volume contexts. There is almost no reason not to.
  
• If you are writing or maintaining a legacy filter, please take the time to make sure that the Group in the INF file is set to the right value. It's a text-only change and it might save a lot of time in support costs...