It's probably pretty clear that FltMgr design shares a lot with IO manager design. In fact, in my opinion, it is a pretty transparent layer. It does not create many abstractions that do not map to IO manager concepts. This is pretty clearly visible in the design of the IO path. One of the FltMgr's main goals was to make better use of kernel stack space and a way to achieve this was to use a callback model as opposed to the IO manager's call-through model, which changes how IO is processed. However, because FltMgr is designed to be as asynchronous as the IO manager, it also makes use of a similar concept to an IRP, which is the FLT_CALLBACK_DATA structure.
Another big difference from the IO manager model is that minifilters can load at any time and at any point in the IO stack. This is pretty significant because when the IO manager allocates an IRP it knows which device it will be sent to and it know the depth of the IO stack and so it knows exactly how many IO_STACK_LOCATIONs that IRP might need. Any new devices will be attached on top of that device and they will not see the IRP. For FltMgr to implement the same model it would need to take a snapshot of the minifilters on the stack at the time each IO is allocated and then only show that IO to those minifilters, skipping any potential minifilters that would be added in between. But what if a minifilter allocates a FLT_CALLBACK_DATA structure to use for issuing IO under certain circumstances ? It is possible that the FLT_CALLBACK_DATA is quite long lived and whenever it will be used some minifilters will be bypassed. So FltMgr was designed in a different way to account for this. Every time a minifilter is done processing a request it returns control to FltMgr, which determines the next node to call at that time. This means that once a minifilter is loaded it might see IO that was initiated before the minifilter was present. It also means that FltMgr cannot know in advance the number of minifilters that will see any particular operation. So while it does keep a stack of structures that similar to the IO_STACK_LOCATION, the array cannot be fixed in size. Instead, FltMgr can reallocate the array on the fly if it discovers that it is about to run out of stack.
The COMPLETION_NODE structure is similar to the IO_STACK_LOCATION and it serves a similar purpose. It is used to know which minifilters to call during IO completion. By default there is an array of such structures that is allocated after the main IRP_CTRL structure header but as I explained above, if FltMgr runs out of entries while processing an operation, it can allocate a new array on the fly and use that one. So let's take a look in the debugger (broken in a preCreate callback for a slightly modified passthrough sample). I've tried to use different colors since I'm a guy and I hope it won't be too difficult to track which fields are related:
I've highlighted a couple of interesting things:1: kd> kn # ChildEBP RetAddr 00 9a7b98d4 96029aeb PassThrough!PtPreOperationPassThrough+0x5b [c:\temp1\passthrough\passthrough.c @ 672] 01 9a7b9940 9602c9f0 fltmgr!FltpPerformPreCallbacks+0x34d 02 9a7b9958 960401fe fltmgr!FltpPassThroughInternal+0x40 03 9a7b996c 960408b7 fltmgr!FltpCreateInternal+0x24 04 9a7b99b0 828744bc fltmgr!FltpCreate+0x2c9 05 9a7b99c8 82a786ad nt!IofCallDriver+0x63 06 9a7b9aa0 82a5926b nt!IopParseDevice+0xed7 07 9a7b9b1c 82a7f2d9 nt!ObpLookupObjectName+0x4fa 08 9a7b9b78 82a9acfa nt!ObOpenObjectByName+0x165 09 9a7b9d24 8287b44a nt!NtQueryAttributesFile+0x121 0a 9a7b9d24 774764f4 nt!KiFastCallEntry+0x12a 1: kd> !fltkd.cbd 0x9239ee40 IRP_CTRL: 9239ede0 CREATE (0) [00000009] Irp SystemBuffer Flags : [1000000c] DontCopyParms Synchronize FixedAlloc Irp : 94452248 DeviceObject : 92fa1c68 "\Device\HarddiskVolume2" FileObject : 92f93558 CompletionNodeStack : 9239ee98 Size=5 Next=1 SyncEvent : (9239edf0) InitiatingInstance : 00000000 Icc : 9a7b9984 CreateIrp.NameCacheCtrl : 9410ce18 CreateIrp.SavedFsContext : 00000000 CallbackData : (9239ee40) Flags : [00000009] Irp SystemBuffer Thread : 942b6d48 Iopb : 9239ee6c RequestorMode : [01] UserMode IoStatus.Status : 0x00000000 IoStatus.Information : 00000000 TagData : 00000000 FilterContext[0] : 00000000 FilterContext[1] : 00000000 FilterContext[2] : 00000000 FilterContext[3] : 00000000 Cmd IrpFl OpFl CmpFl Instance FileObjt Completion-Context Node Adr --------- -------- ----- ----- -------- -------- ------------------ -------- [0,0] 00000000 00 0000 00000000 00000000 00000000-00000000 9239efb8 Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000 [0,0] 00000000 00 0000 00000000 00000000 00000000-00000000 9239ef70 Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000 [0,0] 00000000 00 0000 00000000 00000000 00000000-00000000 9239ef28 Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000 [0,0] 00000000 00 0000 00000000 00000000 00000000-00000000 9239eee0 Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000 [0,0] 00000884 00 0000 923a5678 92f93558 a0ede180-00000000 9239ee98 ("PassThrough","PassThrough Instance") PassThrough!PtPostOperationPassThrough Args: 9a7b99ec 01200000 00070000 00000000 00000000 0000000000000000 Working IOPB: >[0,0] 00000884 00 923a5678 92f93558 9239ee6c ("PassThrough","PassThrough Instance") Args: 9a7b99ec 01200000 00070000 00000000 00000000 0000000000000000 1: kd> dt 9239ede0 fltmgr!_IRP_CTRL +0x000 Type : _FLT_TYPE +0x004 Flags : 0x1000000c (No matching name) +0x008 MajorFunction : 0 '' +0x009 Reserved0 : 0 '' +0x00a CompletionStackLength : 0x5 '' +0x00b NextCompletion : 0x1 '' +0x00c CompletionStack : 0x9239ee98 _COMPLETION_NODE +0x010 SyncEvent : _KEVENT +0x020 Irp : 0x94452248 _IRP +0x020 FsFilterData : 0x94452248 _FS_FILTER_CALLBACK_DATA +0x024 AsyncCompletionRoutine : (null) +0x028 AsyncCompletionContext : (null) +0x02c InitiatingInstance : (null) +0x030 PendingCallbackNode : 0x923a578c _CALLBACK_NODE +0x030 StartingCallbackNode : 0x923a578c _CALLBACK_NODE +0x034 preOp :+0x034 postOp : +0x038 PostCompletionRoutine : 0x96043f3c void fltmgr!FltDeletePushLock+0 +0x03c DeviceObject : 0x92fa1c68 _DEVICE_OBJECT +0x040 FileObject : 0x92f93558 _FILE_OBJECT +0x044 FltWork : _FLTP_WORKITEM +0x044 PendingCallbackContext : (null) +0x048 CachedCompletionNode : (null) +0x04c PendingStatus : 0n0 +0x058 CreateIrp : +0x058 CloseIrp : +0x060 Data : _FLT_CALLBACK_DATA +0x08c WorkingParameters : _FLT_IO_PARAMETER_BLOCK 1: kd> ?? Data struct _FLT_CALLBACK_DATA * 0x9239ee40 +0x000 Flags : 9 +0x004 Thread : 0x942b6d48 _KTHREAD +0x008 Iopb : 0x9239ee6c _FLT_IO_PARAMETER_BLOCK +0x00c IoStatus : _IO_STATUS_BLOCK +0x014 TagData : (null) +0x018 QueueLinks : _LIST_ENTRY [ 0x0 - 0x0 ] +0x020 QueueContext : [2] (null) +0x018 FilterContext : [4] (null) +0x028 RequestorMode : 1 '' 1: kd> dt 9239ee98 fltmgr!_COMPLETION_NODE +0x000 IrpCtrl : 0x9239ede0 _IRP_CTRL +0x004 CallbackNode : 0x923a578c _CALLBACK_NODE +0x004 Filter : 0x923a578c _FLT_FILTER +0x008 InstanceLink : _LIST_ENTRY [ 0x0 - 0x0 ] +0x010 InstanceTrackingList : (null) +0x014 Context : (null) +0x018 DataSnapshot : _FLT_IO_PARAMETER_BLOCK +0x044 Flags : 0 1: kd> ?? sizeof(fltmgr!_IRP_CTRL) + 0x9239ede0 unsigned int 0x9239ee98 1: kd> ??0x9239ede0+0x8c unsigned int 0x9239ee6c
- CompletionNodeStack points to exactly the end of the IRP_CTRL structure
- the Data parameter for the preOp callback is actually a part of the IRP_CTRL structure
- Data->Iopb is actually pointing to the WorkingParameters structure of the IRP_CTRL structure, just as !fltkd.cbd shows for "Working IOPB"
So now we can see how things changed from pre to post:1: kd> !fltkd.cbd @@(Data) IRP_CTRL: 9239ede0 CREATE (0) [00080009] Irp PostOperation SystemBuffer Flags : [1000000c] DontCopyParms Synchronize FixedAlloc Irp : 94452248 DeviceObject : 92fa1c68 "\Device\HarddiskVolume2" FileObject : 92f93558 CompletionNodeStack : 9239ee98 Size=5 Next=1 SyncEvent : (9239edf0) InitiatingInstance : 00000000 SwappedBufferMdl : 9a7b9984 StartingCallbackNode : ffffffff CreateIrp.NameCacheCtrl : 9410ce18 CreateIrp.SavedFsContext : 00000000 CallbackData : (9239ee40) Flags : [00080009] Irp PostOperation SystemBuffer Thread : 942b6d48 Iopb : 9239eeb0 RequestorMode : [01] UserMode IoStatus.Status : 0x00000000 IoStatus.Information : 00000001 TagData : 00000000 FilterContext[0] : 00000000 FilterContext[1] : 00000000 FilterContext[2] : 00000000 FilterContext[3] : 00000000 Cmd IrpFl OpFl CmpFl Instance FileObjt Completion-Context Node Adr --------- -------- ----- ----- -------- -------- ------------------ -------- [0,0] 00000000 00 0000 00000000 00000000 00000000-00000000 9239efb8 Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000 [0,0] 00000000 00 0000 00000000 00000000 00000000-00000000 9239ef70 Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000 [0,0] 00000000 00 0000 00000000 00000000 00000000-00000000 9239ef28 Args: 00000000 00000000 00000000 00000000 00000000 0000000000000000 [0,0] 00000884 00 0000 930a8978 92f93558 96062d6c-49c0c357 9239eee0 ("FileInfo","FileInfo") fileinfo!FIPostCreateCallback Args: 9a7b99ec 01200000 00070000 00000000 00000000 0000000000000000 >[0,0] 00000884 00 0000 923a5678 92f93558 a0ede180-a4e70d1c 9239ee98 ("PassThrough","PassThrough Instance") PassThrough!PtPostOperationPassThrough Args: 9a7b99ec 01200000 00070000 00000000 00000000 0000000000000000 Working IOPB: [0,0] 00000884 00 930a8978 92f93558 9239ee6c ("FileInfo","FileInfo") Args: 9a7b99ec 01200000 00070000 00000000 00000000 0000000000000000 1: kd> ?? Data struct _FLT_CALLBACK_DATA * 0x9239ee40 +0x000 Flags : 0x80009 +0x004 Thread : 0x942b6d48 _KTHREAD +0x008 Iopb : 0x9239eeb0 _FLT_IO_PARAMETER_BLOCK +0x00c IoStatus : _IO_STATUS_BLOCK +0x014 TagData : (null) +0x018 QueueLinks : _LIST_ENTRY [ 0x0 - 0x0 ] +0x020 QueueContext : [2] (null) +0x018 FilterContext : [4] (null) +0x028 RequestorMode : 1 '' 1: kd> dt 9239ee98 fltmgr!_COMPLETION_NODE +0x000 IrpCtrl : 0x9239ede0 _IRP_CTRL +0x004 CallbackNode : 0x923a578c _CALLBACK_NODE +0x004 Filter : 0x923a578c _FLT_FILTER +0x008 InstanceLink : _LIST_ENTRY [ 0x944ffa04 - 0x9449eea0 ] +0x010 InstanceTrackingList : 0x944ffa00 _COMPLETION_NODE_TRACKING_LIST +0x014 Context : 0xa4e70d1c Void +0x018 DataSnapshot : _FLT_IO_PARAMETER_BLOCK +0x044 Flags : 0 1: kd> ?? 0x9239ee98+0x18 unsigned int 0x9239eeb0
- The most important change is that while Data is at the same location (since it is allocated in the IRP_CTRL) but Data->Iopb points to the DataSnapshot member of the COMPLETION_NODE structure.
- !fltkd.cbd still shows WorkingIOPB as being 9239ee6c when in fact it should be 0x9239eeb0. So don't trust the WorkingIOPB in a postOp callback. However, the Iopb member under CallbackData is actually correct, pointing to the right Iopb.
- WorkingIOPB looks like the IOPB for the COMPLETION_NODE for the fileinfo minifilter, the lowest one in this frame.
PreOP
- before calling a preOp callback fltmgr looks at whether the minfilter has a postOp callback registered and if so it initializes a COMPLETION_NODE structure on the stack and copies WorkingParameters into the IOPB member.
- IRP_CTRL->Data->Iopb is set to point to IRP_CTRL->WorkingParameters (this is the same address for all preOp callbacks)
- preOp callback is called
- if preOp callback doesn't want a postOp callback, then the COMPLETION_NODE on the stack is "released"
- any changes the minifilter made to the IOPB were made directly into WorkingParameters where they might be copied in the COMPLETION_NODE for the next minifilter
- if the minifilter wanted a postOp callback, FltMgr will add the COMPLETION_NODE to a list of COMPLETION_NODEs to be drained if the minifilter unloads.
PostOP
- before calling the postOp callback fltmgr changes the IRP_CTRL->Data->Iopb to point to the COMPLETION_NODE->DataSnapshot
- postOp is called
- after postOp returns the CompletionNodeStack is changed to point to the next (or previous depending on your perspective; the node for the first minifilter with a higher altitude that wanted a postOp callback) COMPLETION_NODE to be called.
- FltMgr never looks at the IOPB to see if the filter changed something. It also doesn't look at the FLTFL_CALLBACK_DATA_DIRTY. The parameters are always saved before the preOp callback is called if the minifilter has a postOp callback. The next minifilter down gets exactly the same IOPB location (in the preOp path). This is not exactly the same as the IO_STACK_LOCATION mechanism.
- In the postOp callback there is no point to changing the IOPB, it will be discarded and any change will be ignored. This also means that if a minifilter does change the IOPB it doesn't need to call FltSetCallbackDataDirty() in a postOp.
Posts
+1. Nice article.
ReplyDelete