An Extension for Tracking Stack Usage in Drivers

The kernel stack is a very limited resource and drivers need to be very careful about how they use that stack. File system filters in particular need to be very frugal, since they are in the IO path and any IO requests they make will use whatever stack remains. Then there is the issue of reentrancy which might not be such a serious issue for minifilters, but it still matters (there are still some cases where name provider callbacks are involved where stack can disappear quite quickly). Also, with the number of minifilters on the rise it is clear that stack usage is something file system filter developers must be extra careful about.

So as I was looking into reducing stack usage for one of my projects I (which I knew had some issues with stack usage) I realized that I didn't have a good way to know which functions to look at. So I finally started on a project I've been meaning to work on for a while but never had the time. The result of that is a debugger extension that looks at a set of functions (matching a pattern) and displays the size of the local variables for each function. Here are some examples :

0: kd> !ace.stack_usage passthrough!*
96 bytes used for locals by func: a5dee450 PassThrough!PtNormalizeNameComponentExCallback (...)
48 bytes used for locals by func: a5dee250 PassThrough!PtGenerateFileNameCallback (...)
20 bytes used for locals by func: a5dee620 PassThrough!_except_handler4 (...)
16 bytes used for locals by func: a5dee020 PassThrough!PtPreOperationPassThrough (...)
12 bytes used for locals by func: a5df2010 PassThrough!DriverEntry (...)
8 bytes used for locals by func: a5dee0d0 PassThrough!PtOperationStatusCallback (...)
8 bytes used for locals by func: a5dee1d0 PassThrough!PtDoRequestOperationStatus (...)
4 bytes used for locals by func: a5df1010 PassThrough!PtInstanceSetup (...)
4 bytes used for locals by func: a5df1150 PassThrough!PtInstanceTeardownComplete (...)
4 bytes used for locals by func: a5df10f0 PassThrough!PtInstanceTeardownStart (...)

Probably the biggest limitation is that it requires an x86 target machine which isn't a problem at the moment since most drivers have an x86 version anyway. I'm working on doing this for amd64 but it requires a pretty different implementation. One interesting aspect of it is that it also works with public symbols and so you can actually see stack usage for the top fltmgr functions for example:

0: kd> !ace.stack_usage fltmgr!* 20
524 bytes used for locals by func: 9626cda0 fltmgr!FltpvPrintErrors = 
404 bytes used for locals by func: 9625fff0 fltmgr!FltpCalculateLegacyFilterAltitude = 
384 bytes used for locals by func: 96262be2 fltmgr!FltpDoUnloadFilter = 
356 bytes used for locals by func: 96260ca0 fltmgr!FltpOpenInstancesRegistryKey = 
332 bytes used for locals by func: 96262ddc fltmgr!FltLoadFilter = 
300 bytes used for locals by func: 962672c0 fltmgr!FltGetDestinationFileNameInformation = 
292 bytes used for locals by func: 9624a9d0 fltmgr!DrainCompletionNode = 
280 bytes used for locals by func: 9625f558 fltmgr!FltpFsNotificationActual = 
184 bytes used for locals by func: 9625f3a6 fltmgr!FltpLogEvent = 
148 bytes used for locals by func: 9625c4b4 fltmgr!FltpGetVolumeFromName = 
144 bytes used for locals by func: 9624dc20 fltmgr!FltSendMessage = 
124 bytes used for locals by func: 96273506 fltmgr!FltpInitializeMessagingSupport = 
112 bytes used for locals by func: 96273878 fltmgr!FltpBuildParameterOffsetTable = 
100 bytes used for locals by func: 9626295a fltmgr!FltpOpenLinkOrRenameTarget = 
100 bytes used for locals by func: 9627316e fltmgr!DriverEntry = 
96 bytes used for locals by func: 96263694 fltmgr!FltpOpenClientPort = 
96 bytes used for locals by func: 9625d174 fltmgr!FltpEnumerateAggregateFilterInformation = 
92 bytes used for locals by func: 9626a522 fltmgr!FltGetVolumeGuidName = 
88 bytes used for locals by func: 9624d1fa fltmgr!FltReadFile = 
88 bytes used for locals by func: 9626d5a6 fltmgr!FltvPreOperation = 

You can download the file at https://sites.google.com/site/fsfilters/downloads/ACE1.0.zip?attredirects=0&d=1 and give it a go. I hope you'll find it useful, I've had a couple of surprises when running it on my drivers :)...