Archive for January, 2008

OSR Seminars at Home

Tuesday, January 15th, 2008

If one of the problems that had prevented you from making a specialized training in developing device drivers at the OSR was a fear of flying, so there is no problem anymore. OSR, after years and years of experience teaching how to write drivers, has now invested in Webinars. This news has already been announced since last Jan. 4, but it is still surprising when I comment it with some people. So, I have done this post trying to pass it to more people at once.

 

 

Cheaper?

The financial advantage is in the fact of avoiding buying flying tickets and paying hosting for a week in the United States. Okay, okay, it’s that’s not you who is going to  pay it, but figure out that not everyone works for multi-national companies with thousands of dollars earmarked for employee training. When I had done the training at OSR, everything was paid by the company which I used to work for; however,  if I put everything on the pencil, it was approximately R$ 13,000.oo of investment to attend a seminar of U$ 2,350.00. Taking the current dollar exchange rate, it would be approximately R$ 4,143.00, or roughly, a third of the total. This may simply be the factor that is preventing you from doing the training. So, you can go running to Mommy and Daddy to say them that now they have more purchasing power for your training. But, not three times more. It seems to me Webinars are relatively more expensive than the conventional ones. The seminar I had attended lasted for 32 hours and addressed the File System Driver development (a little advanced subject), while the first webinar offered by OSR will is suppose to do a quick introduction to WDF for 20 hours for the same price.

More Advantages

Another very relevant point is the fact that you don’t need to leave home or the company  that you work for during the seminar. Some people whom I talked to about it told me that a big problem is the fact of leaving home for a week. Some of them have small children or many other reasons you might imagine, but I think the most common reason holding a professional at work is the endless urgent bugs which can only be solved by him. Well, at least this webinar consists of 5 lessons of 4 hours. This allows you to take care of your bug collection while participating into the trainings.

It is not only an expensive PPT

Contrary to what one might figure out, the seminars are going to be interactive using two-way connections where the participant may, in addition to receiving all instructions live, ask questions and even receive all the printed material like in other seminars. The student is going to download the tool that establishes the connection and attend a 15-minute session to test the machine to be used in them and thus, make sure that the participant is having no problem during the training.

And now, if your company really does not want to pay training for you, it is, at least, easier to pay it for yourself; this way you may use this as a new star on your resume.
Goodbye!

CLEANUP and CLOSE

Wednesday, January 9th, 2008

I have written a post that describes the steps of how to create a simple driver. This driver simply keeps a list of strings it receives via writing IRPs and returns them via reading IRPs. Well, if you do not know what an IRP is, another post tries to explain what they are and even describes the needed steps to use the IRP Tracker to observe the IRPs coming and going. If we do a little test using the IRP Tracker on the sample driver that I have just commented, we will have an output similar to the figure below.


We can see all the IRPs that our device has received from IRP_MJ_CREATE to IRP_MJ_CLOSE. Among these IRPs, we can notice that some of them were not completed successfully. Because no routine has been designed to treat IRP_MJ_CLEANUP, these IRPs are completed with STATUS_INVALID_DEVICE_REQUEST. In this post I’m going to talk a little about this IRP and how your device interacts with the processes which have obtained a handle for it.

To see the IRP details, double-click on the IRP line of the IRP Tracker so that the window below can appear. In this window you can see, among the other details, the FileObject used in the IRP. This is helping us in this post during the tests.

When does an IRP_MJ_CLEANUP take place?

The Object Manager sends an CLEANUP IRP in order to notify the driver that the last handle for a given FileObject has been closed. As some of you may know, when an application uses CreateFile() to get a handle to our device, this results in a FileObject creation. The subsequent operations using this handle will be linked to this FileObject. More details about this are going to be found in this post.

A FileObject does not have a direct relation with a handle. A handle might be duplicated  or even inherited from the parent process on creating a new process. The result of these actions is having multiple handles to be translated into the same FileObject. Thus, not always when an application calls the CloseHandle function, a CLEANUP or CLOSE IRP is sent to the driver.

To follow the steps below, the sample program source code will be available for downloading at the end of this post. This program uses the example driver that have been built in another post; the driver source code can be downloaded from here. From the sources, you can compile and install the test driver. If you do not know how to do this, this post can help you. After installing the test driver, it would be interesting to execute the calls below with the help of a debugger, and thus, be able to observe the results on the IRP Tracker for ever executed line.

It’s important to read the source comments below. I got lazy to duplicate this information here in the post.

/****
***     main
**
**      I hope that everyone know this is the
**      application entrypoint. Otherwise,
**      you might be precipitaded on reading a
**      Windows driver blog.
*/
int main(int argc, char* argv[])
{
    HANDLE  h1, h2, h3;
    CHAR    szBuffer[100];
    DWORD   dwBytes;
 
    //-f--> Here we are opening the first handle
    //      to our device. That is the step 1
    //      on the IRP Tracker. Check out the FileObject
    //      value so that you can compare it in
    //      future requests.
    h1 = CreateFile("\\\\.\\EchoDevice",
                    GENERIC_ALL,
                    0,
                    NULL,
                    OPEN_EXISTING,
                    0,
                    NULL);
 
    //-f--> Here we are opening the second handle
    //      to our device. That is the step 2
    //      on the IRP Tracker. Check out the FileObject
    //      value so that you can compare it in
    //      future requests.
    h2 = CreateFile("\\\\.\\EchoDevice",
                    GENERIC_ALL,
                    0,
                    NULL,
                    OPEN_EXISTING,
                    0,
                    NULL);
 
    //-f--> Here we are throwing an IRP_MJ_READ for the
    //      first FileObject we have gotten. Step 3 on
    //      IRP Tracker. Notice that the first FileObject
    //      is going to be used in this IRP.
    ReadFile(h1,
             szBuffer,
             sizeof(szBuffer),
             &dwBytes,
             NULL);
 
    //-f--> Here we are throwing an IRP_MJ_READ for the
    //      second FileObject we have gotten. Step 4 on
    //      IRP Tracker. Notice that the second FileObject
    //      is going to be used in this IRP.
    ReadFile(h2,
             szBuffer,
             sizeof(szBuffer),
             &dwBytes,
             NULL);
 
    //-f--> The handle duplication is not notified
    //      to the driver. Only Object Manager "has known"
    //      about that. There isn't any corresponding
    //      step on IRP Tracker.
    DuplicateHandle(GetCurrentProcess(),
                    h1,
                    GetCurrentProcess(),
                    &h3,
                    0,
                    FALSE,
                    DUPLICATE_SAME_ACCESS);
 
    //-f--> As the third handle was goten from the
    //      first handle duplication, the corresponding
    //      FileObject is the same as the first handle.
    //      Step 5 on IRP Tracker. Notice that the first
    //      FileObject is going to be used on this IRP.
    ReadFile(h3,
             szBuffer,
             sizeof(szBuffer),
             &dwBytes,
             NULL);
 
    //-f--> Because we have two handles for the first FileObject,
    //      closing one of them will not gererate any notification
    //      to our device. There isn't any corresponding step
    //      on IRP Tracker.
    CloseHandle(h1);
 
    //-f--> That handle has not been duplicated, thus, when it is closed,
    //      an IRP_MJ_CLEANUP followed by an IRP_MJ_CLOSE are going to be
    //      sent to the driver. Step 6 on IRP Tracker.
    CloseHandle(h2);
 
    //-f--> Closing that handle, we will have the same behavior
    //      seen during the h2 closing. From the driver
    //      viewpoint, the first FileObject will be destroyed
    //      now. Step 7 on IRP Tracker.
    CloseHandle(h3);
 
    //-f--> And they have all lived happy ever after.
    return 0;
}

Getting a handle to an object assures us that this object will be valid until we close this handle. An object can only be destroyed by the system after all handles to it are closed. For this, the Object Manager maintains two counters, ProcessHandleCount and SystemHandleCount. The first one keeps the amount of open handles to an object in a given process. The other one maintains the sum of all ProcessHandleCount for the object in the system. These counters are decremented as these handles are closed. When they reach zero, a IRP_MJ_CLEANUP is generated.

Usually an IRP_MJ_CLEANUP serves us as an event to cancel any asynchronous operation on the FileObject which is being finalized. These IRPs are linked to the threads that have launched them and any asynchronous IRP should be cancelled at this time.

But what is IRP_MJ_CLOSE used for?

Besides the reference counters above mentioned, there is also the ObjectReferenceCount which, besides being incremented when a new handle is obtained, it is also incremented when a reference is made using kernel functions like ObReferenceObject(), for example. These calls increment the ObjectReferenceCount without a new handle being generated. For those who have known COM, this call has the similar behavior to AddRef(). This allows the object to remain valid itself for the kernel, even after all handles to it have been destroyed. Anyway, when this counter reaches zero, then the IRP_MJ_CLOSE is sent.

A driver can associate data structures for a given FileObject using the FsContext and FsContext2 pointers according to what I had  discussed in another post. These structures can only be deallocated when the driver receives the IRP_MJ_CLOSE.

Operations after IRP_MJ_CLEANUP

When the driver receives an IRP_MJ_CLEANUP, it does not mean that the end is near. Other kernel components can still launch new IRPs for reading or writing, even after this event. Not to mention the IRP_MJ_INTERNAL_DEVICE_CONTROL that can be exchanged among drivers.

A very common scenario at the File Systems development is precisely about a reference to a FileObject that is maintained by the Cache Manager. Even when all handles are closed by the applications, the system still retains this refrence anticipating that some application may want to open the same file again. The Cache Manager has some system threads that perform the so-called “delayed writing”. This feature retains many writes to a file in a single operation aftermost in order to reduce the number of disk accesses, thus gaining performance. It is too often such writings taking place, once the file had all of its handles closed for applications, and then, these data being written after IRP_MJ_CLEANUP.

In the end of it all, I hope to once again have helped more than hindered. Issues regarding the Cache Manager, Memory Manager and File Systems are not very trivial, but they are interesting enough to be read about and understand what the system has done before saying that everything is crap.

Until next time!

TestCleanup.zip

The wrist is still pulsing

Friday, January 4th, 2008

Before completing three months without any sign of life, here I am giving a peek at what has happened at the blogosphere. This year ending has been really hectic. As you may know, my employment by IBM has contributed a little to my absence. Okay, the university has also helped a lot. At the very end of the year, I reserved a time for myself and went for a walk on the beach. But let’s stop talking and get to what matters.

In this post, back to the living world, I am going to only comment on small things and postpone for a more elaborated post later (but still in this life), like you’ve been used to seeing around here.

Debug in Free

One of the things I had to learn to live together at IBM was to debug the driver we were supposed to write, which had always been built in Free or with optimizations and everything else that a release building deserved. Well, I have tried reluctantly to say it would be important to have a checked version for testing and that would be valuable to run our driver with all ASSERTs turned on check anything unusual that might occur. But believe me, it is not that easy to convince people that have already being accustomed to this situation. I was looking for good reasons to have a usable checked version when I had found this post. Well, the reasons are actually quite good but I still argue that we should have a checked version . I have been preparing a post that says a little about it.

Talking about knowing English

At some posts before this, I had talked about MSDN translated into some languages, including Portuguese. The message was that we could deal with it a little without knowing English but, if you don’t know English and you want to develop drivers for a living, so you have to learn English as soon as possible. I remembered this post today at a Google search when a link was especially rewarding.

IoGetDeviceObjectPointer Tips

Well, if someone has managed to get some tips, please tell me. If one day I decide to learn a third language, now I have a candidate for it.

New Year, New Look

It was supposed to have happened before, but unfortunately the time is has been short for many people. I put in check brother’s web designer skills. I asked him to improve this blog layout. It’s getting pretty good, but I still have had nothing concrete to be shown. I will be migrating to WordPress soon. Most of the work has already been done but much of it has remained to be done. My main intention in migrating to WordPress is to be able to use one of these plug-ins that would allow me to have the same post in both languages, English and Portuguese. Well, we have tried to test a few of them but they got some problems. But we’ll get there.

CYA…