CLEANUP and CLOSE

January 9th, 2008 - Fernando Roberto

I have written a post that describes the steps of how to create a simple driver. This driver simply keeps a list of strings it receives via writing IRPs and returns them via reading IRPs. Well, if you do not know what an IRP is, another post tries to explain what they are and even describes the needed steps to use the IRP Tracker to observe the IRPs coming and going. If we do a little test using the IRP Tracker on the sample driver that I have just commented, we will have an output similar to the figure below.


We can see all the IRPs that our device has received from IRP_MJ_CREATE to IRP_MJ_CLOSE. Among these IRPs, we can notice that some of them were not completed successfully. Because no routine has been designed to treat IRP_MJ_CLEANUP, these IRPs are completed with STATUS_INVALID_DEVICE_REQUEST. In this post I’m going to talk a little about this IRP and how your device interacts with the processes which have obtained a handle for it.

To see the IRP details, double-click on the IRP line of the IRP Tracker so that the window below can appear. In this window you can see, among the other details, the FileObject used in the IRP. This is helping us in this post during the tests.

When does an IRP_MJ_CLEANUP take place?

The Object Manager sends an CLEANUP IRP in order to notify the driver that the last handle for a given FileObject has been closed. As some of you may know, when an application uses CreateFile() to get a handle to our device, this results in a FileObject creation. The subsequent operations using this handle will be linked to this FileObject. More details about this are going to be found in this post.

A FileObject does not have a direct relation with a handle. A handle might be duplicated  or even inherited from the parent process on creating a new process. The result of these actions is having multiple handles to be translated into the same FileObject. Thus, not always when an application calls the CloseHandle function, a CLEANUP or CLOSE IRP is sent to the driver.

To follow the steps below, the sample program source code will be available for downloading at the end of this post. This program uses the example driver that have been built in another post; the driver source code can be downloaded from here. From the sources, you can compile and install the test driver. If you do not know how to do this, this post can help you. After installing the test driver, it would be interesting to execute the calls below with the help of a debugger, and thus, be able to observe the results on the IRP Tracker for ever executed line.

It’s important to read the source comments below. I got lazy to duplicate this information here in the post.

/****
***     main
**
**      I hope that everyone know this is the
**      application entrypoint. Otherwise,
**      you might be precipitaded on reading a
**      Windows driver blog.
*/
int main(int argc, char* argv[])
{
    HANDLE  h1, h2, h3;
    CHAR    szBuffer[100];
    DWORD   dwBytes;
 
    //-f--> Here we are opening the first handle
    //      to our device. That is the step 1
    //      on the IRP Tracker. Check out the FileObject
    //      value so that you can compare it in
    //      future requests.
    h1 = CreateFile("\\\\.\\EchoDevice",
                    GENERIC_ALL,
                    0,
                    NULL,
                    OPEN_EXISTING,
                    0,
                    NULL);
 
    //-f--> Here we are opening the second handle
    //      to our device. That is the step 2
    //      on the IRP Tracker. Check out the FileObject
    //      value so that you can compare it in
    //      future requests.
    h2 = CreateFile("\\\\.\\EchoDevice",
                    GENERIC_ALL,
                    0,
                    NULL,
                    OPEN_EXISTING,
                    0,
                    NULL);
 
    //-f--> Here we are throwing an IRP_MJ_READ for the
    //      first FileObject we have gotten. Step 3 on
    //      IRP Tracker. Notice that the first FileObject
    //      is going to be used in this IRP.
    ReadFile(h1,
             szBuffer,
             sizeof(szBuffer),
             &dwBytes,
             NULL);
 
    //-f--> Here we are throwing an IRP_MJ_READ for the
    //      second FileObject we have gotten. Step 4 on
    //      IRP Tracker. Notice that the second FileObject
    //      is going to be used in this IRP.
    ReadFile(h2,
             szBuffer,
             sizeof(szBuffer),
             &dwBytes,
             NULL);
 
    //-f--> The handle duplication is not notified
    //      to the driver. Only Object Manager "has known"
    //      about that. There isn't any corresponding
    //      step on IRP Tracker.
    DuplicateHandle(GetCurrentProcess(),
                    h1,
                    GetCurrentProcess(),
                    &h3,
                    0,
                    FALSE,
                    DUPLICATE_SAME_ACCESS);
 
    //-f--> As the third handle was goten from the
    //      first handle duplication, the corresponding
    //      FileObject is the same as the first handle.
    //      Step 5 on IRP Tracker. Notice that the first
    //      FileObject is going to be used on this IRP.
    ReadFile(h3,
             szBuffer,
             sizeof(szBuffer),
             &dwBytes,
             NULL);
 
    //-f--> Because we have two handles for the first FileObject,
    //      closing one of them will not gererate any notification
    //      to our device. There isn't any corresponding step
    //      on IRP Tracker.
    CloseHandle(h1);
 
    //-f--> That handle has not been duplicated, thus, when it is closed,
    //      an IRP_MJ_CLEANUP followed by an IRP_MJ_CLOSE are going to be
    //      sent to the driver. Step 6 on IRP Tracker.
    CloseHandle(h2);
 
    //-f--> Closing that handle, we will have the same behavior
    //      seen during the h2 closing. From the driver
    //      viewpoint, the first FileObject will be destroyed
    //      now. Step 7 on IRP Tracker.
    CloseHandle(h3);
 
    //-f--> And they have all lived happy ever after.
    return 0;
}

Getting a handle to an object assures us that this object will be valid until we close this handle. An object can only be destroyed by the system after all handles to it are closed. For this, the Object Manager maintains two counters, ProcessHandleCount and SystemHandleCount. The first one keeps the amount of open handles to an object in a given process. The other one maintains the sum of all ProcessHandleCount for the object in the system. These counters are decremented as these handles are closed. When they reach zero, a IRP_MJ_CLEANUP is generated.

Usually an IRP_MJ_CLEANUP serves us as an event to cancel any asynchronous operation on the FileObject which is being finalized. These IRPs are linked to the threads that have launched them and any asynchronous IRP should be cancelled at this time.

But what is IRP_MJ_CLOSE used for?

Besides the reference counters above mentioned, there is also the ObjectReferenceCount which, besides being incremented when a new handle is obtained, it is also incremented when a reference is made using kernel functions like ObReferenceObject(), for example. These calls increment the ObjectReferenceCount without a new handle being generated. For those who have known COM, this call has the similar behavior to AddRef(). This allows the object to remain valid itself for the kernel, even after all handles to it have been destroyed. Anyway, when this counter reaches zero, then the IRP_MJ_CLOSE is sent.

A driver can associate data structures for a given FileObject using the FsContext and FsContext2 pointers according to what I had  discussed in another post. These structures can only be deallocated when the driver receives the IRP_MJ_CLOSE.

Operations after IRP_MJ_CLEANUP

When the driver receives an IRP_MJ_CLEANUP, it does not mean that the end is near. Other kernel components can still launch new IRPs for reading or writing, even after this event. Not to mention the IRP_MJ_INTERNAL_DEVICE_CONTROL that can be exchanged among drivers.

A very common scenario at the File Systems development is precisely about a reference to a FileObject that is maintained by the Cache Manager. Even when all handles are closed by the applications, the system still retains this refrence anticipating that some application may want to open the same file again. The Cache Manager has some system threads that perform the so-called “delayed writing”. This feature retains many writes to a file in a single operation aftermost in order to reduce the number of disk accesses, thus gaining performance. It is too often such writings taking place, once the file had all of its handles closed for applications, and then, these data being written after IRP_MJ_CLEANUP.

In the end of it all, I hope to once again have helped more than hindered. Issues regarding the Cache Manager, Memory Manager and File Systems are not very trivial, but they are interesting enough to be read about and understand what the system has done before saying that everything is crap.

Until next time!

TestCleanup.zip

Leave a Reply