DriverEntry

We want samples
A few months after my hiring into Open, they asked me to do a participation in a lecture about Secure Code. My part was about stack overflow and how to take advantage of that oversight programmer to break into the program. The major point to note was in addition to programmers, the audience was composed by software engineers and architects, the commercial staff, who had more contact with customers were there also, there were one or two people from administrative sector as well, in summary, an audience that few of them have heard about call stack. One of them said: “I sort of remember from when I got my .Net certification, which said that structures are created on the heap and objects are created on the heap, or vice versa.” Anyway, things became more complicated when I started to show the sample code I supplied, with PUSHs, MOVs and POPs. The result was: some slept, while others just drooled and spoke in that alien language while they sleep. I know that language very well because my wife always talks to me while she sleeps and I’m sitting in bed with the notebook. Despite her words being completely incomprehensible, she always responds when I make questions to her. She introduces the subject by saying: “Admivoza bumizav” then I ask: “Why do you think so?” And she replies: “Zumirag abmish mua”. But back to the subject, it became clear that the amount of technical detail was incompatible with the public. So, not long ago, in a talking about Windows drivers, I tried to pass a vision with little details, just to give an idea what drivers are and how they contribute to the system work. After all, the entire development department was there, including architects, engineers and .Net coders (nothing against them). To my surprise, at the end of the lecture, everyone was expecting more details, something like: “Hey, what about the remaining? Don’t you have any little example?”. So, okay… In this post, I will write a minimal driver, but one that has some interaction with a test application.

At the beginning…

Here I will assume that you have already known how to create a driver project from scratch and how to use Visual Studio to code drivers, going straight to the point where we must write the driver. The today’s example will be a very basic echo driver commands that will receive commands through reading and writing. So, it would be like that: you initially write buffers, which are strings in our example using the WriteFile() function and everything will be stored into the driver. The subsequent readings from the function ReadFile() will bring the same data from where were written. This example will be very useful in other posts, and will also serve as a starting point for those wanting to write their first driver.

Knowing that we will store the strings in a list, and then we need to create a buffer list composed by nodes defined as shown below.
```
//-f--> Type definition to be stored into
```
```
//      our buffer list.
```
```
typedef struct _BUFFER_ENTRY
```
```
{
```
```
    PVOID       pBuffer;    //-f--> Sent buffer
```
```
    ULONG       ulSize;     //      Buffer size
```
```
    LIST_ENTRY  Entry;      //      List node
```
```
 
```
```
} BUFFER_ENTRY, *PBUFFER_ENTRY;
```
```
 
```
```
 
```
```
//-f--> List head and mutex for protection 
```
```
LIST_ENTRY      g_BufferList;
```
```
KMUTEX          g_Mutex;
```
After defining the structure, we have to create a global variable that will be the list head and also a mutex, to protect our list from possible accesses in parallel. This would happen if we had two test applications running at the same time. If you want more details about the DDK linked lists, there is a post that explains about that, too.

Writing DriverEntry routine

The first point to notice here is that, since our example has been coded in a .CPP file, it’s necessary to put an extern “C” at the DriverEntry function definition. Otherwise, the linker will not find the driver entry point. Early in the implementation, we can see the message that will be launched to the debugger via KdPrint(). Next, I will initialize our linked list head and the mutex that protect it. Now let’s set some members in the DriverObject structure and the first will be DriverUnload member, which receives a pointer to a callback function that will inform the driver that it is being unloaded. The next few members are the routines that will be called when the driver receives requests about Create/Open, Close, Read and Write. I will talk about these routines with more details a little later. Continuing, the DeviceObject is created, which will be our way of communication way with the driver. As I said at the lecture, all requests a driver receives are through a device. IoCreateDevice() function makes this for us. Created the device, I will now configure it so that it should use intermediate buffers. For this, I have to set the bit DO_BUFFERED_IO in Flags field from DeviceObject you have just created. Intermediate buffers? We’ll talk about BufferedIo versus DirectIo in a future opportunity. There are so many new concepts in this post and I, unfortunately cannot go very deep into their details, otherwise nobody would read it that never seems ending up.

Having a DeviceObject is cool, but it is not everything in driver’s life; for a User Mode application being able to communicate with a driver, you must create a Symbolic Link. This is done next with IoCreateSymbolicLink() function. The remaining function code should not cause great surprise to most of you, but if in doubt, just send me an e-mail and we can resolve this in a fight.
```
/****
```
```
***     DriverEntry
```
```
**
```
```
**      This is the driver entry point.
```
```
**      Knife in teeth and blood in eyes.
```
```
*/
```
```
 
```
```
extern "C"
```
```
NTSTATUS DriverEntry(IN PDRIVER_OBJECT pDriverObj,
```
```
                     IN PUNICODE_STRING pusRegistryPath)
```
```
{
```
```
    NTSTATUS        nts;
```
```
    PDEVICE_OBJECT  pDeviceObj = NULL;
```
```
 
```
```
    __try
```
```
    {
```
```
        //-f--> Saying hello to the Kernel Debugger
```
```
        KdPrint(("Starting KernelEcho driver...\n"));
```
```
 
```
```
        //-f--> Initializing the buffer head list and mutex
```
```
        InitializeListHead(&g_BufferList);
```
```
        KeInitializeMutex(&g_Mutex, 0);
```
```
 
```
```
        //-f--> Setting the driver unload callback
```
```
        pDriverObj->DriverUnload = OnDriverUnload;
```
```
 
```
```
        //-f--> Setting the driver routines that
```
```
        //      will be supported.
```
```
        pDriverObj->MajorFunction[IRP_MJ_CREATE] = OnCreate;
```
```
        pDriverObj->MajorFunction[IRP_MJ_CLOSE] = OnClose;
```
```
        pDriverObj->MajorFunction[IRP_MJ_WRITE] = OnWrite;
```
```
        pDriverObj->MajorFunction[IRP_MJ_READ] = OnRead;
```
```
 
```
```
        //-f--> Creating the control device
```
```
        nts = IoCreateDevice(pDriverObj,
```
```
                             0,
```
```
                             &g_usDeviceName,
```
```
                             FILE_DEVICE_UNKNOWN,
```
```
                             0,
```
```
                             FALSE,
```
```
                             &pDeviceObj);
```
```
        if (!NT_SUCCESS(nts))
```
```
            ExRaiseStatus(nts);
```
```
 
```
```
        //-f--> Configuring I/O using intermediary buffer
```
```
        pDeviceObj->Flags |= DO_BUFFERED_IO;
```
```
 
```
```
        //-f--> Creating a symbolic link, so applications
```
```
        //      can reach this device.
```
```
        nts = IoCreateSymbolicLink(&g_usSymbolicLink,
```
```
                                   &g_usDeviceName);
```
```
        if (!NT_SUCCESS(nts))
```
```
            ExRaiseStatus(nts);
```
```
 
```
```
    }
```
```
    __except(EXCEPTION_EXECUTE_HANDLER)
```
```
    {
```
```
        //-f--> Getting the error code
```
```
        nts = GetExceptionCode();
```
```
 
```
```
        //-f--> That will force the debugger to stop here.
```
```
        //      But only when this code is built in Checked mode.
```
```
        ASSERT(FALSE);
```
```
        KdPrint(("An exception occurred at " __FUNCTION__ "\n"));
```
```
 
```
```
        //-f--> Since we had problems during the initialization, let's
```
```
        //      undo what was done.
```
```
        if (pDeviceObj)
```
```
            IoDeleteDevice(pDeviceObj);
```
```
    }
```
```
 
```
```
    return nts;
```
```
}
```
Writing Dispatch Functions

I’ll also assume that you have already had an idea of what an IRP is. Now let’s write the functions to manipulate them. They are called Dispatch Functions. These functions are set at driver startup time as you have seen in the code above. In the DriverObject structure, the MajorFunction member is a function pointer array indexed for macros like IRP_MJ_READ. The function prototype is the same for all functions and it will be displayed below. Dispatch Function needs to handle IRPs following some rules, like everything else at DDK. A minimal function could be written as follows:
```
/****
```
```
***     OnDispatch
```
```
**
```
```
**      A minimum Dispatch Function example
```
```
*/
```
```
 
```
```
NTSTATUS
```
```
OnDispatch(IN PDEVICE_OBJECT  pDeviceObj,
```
```
           IN PIRP            pIrp)
```
```
{
```
```
    //-f--> Here I fill the IRP status.
```
```
    pIrp->IoStatus.Status = STATUS_SUCCESS;
```
```
    pIrp->IoStatus.Information = 0;
```
```
 
```
```
    //-f--> Completing the IRP. After that, touching the IRP structure
```
```
    //      is absolutely prohibited. That does not belong to you anymore.
```
```
    IoCompleteRequest(pIrp, IO_NO_INCREMENT);
```
```
 
```
```
    //-f--> Return status code to the IoManager.
```
```
    return STATUS_SUCCESS;
```
```
}
```
But what if we had called the ReadFile() function with a handle to our device if we did not fill the position IRP_MJ_READ? Would we burn up forever in marble from hell? Indeed, if we take a look at MajorFunction table before filling it, we can see that there is the same address in all slots. Let’s put a break-point at DriverEntry entrance, take a look on the table slots before being filled out and see what’s there.

As we saw, this table is all initialized with a function pointer that it’s in implementation; considering that the power management IRPs have a special treatment, it completes the IRP with status STATUS_INVALID_DEVICE_REQUEST.

A Dispatch Function basically follows one of the three alternatives for dealing with an IRP. In cases of filters, our driver could pass the IRP to the driver which it was attached to. Okay, that was noted here… “Do a post providing a filter example”. The second alternative would retain the IRP to an asynchronous processing and the last but not least, simply complete the IRP. Notice that in our example, all that we do is to tell the application that the IRP was successfully performed and completed it. To complete the IRP, we use the IoCompleteRequest() function, which receives the IRP to be completed and the Priority Boost. What? Assuming your IRP had some interaction with hardware, it would consume some thread time in Kernel Mode. This time it would create a delay in the current thread and this would be compensated by this Boost. Because this is not our case, we use the macro to define no Boost. DDK has a list of constants that determines the boost that a thread should receive for each type of device. See the WDM.h excerpt (this definition may be in ntddk.h depending on your DDK version).
```
//
```
```
// Priority increment for completing CD-ROM I/O.  This is used by CD-ROM device
```
```
// and file system drivers when completing an IRP (IoCompleteRequest)
```
```
//
```
```
 
```
```
#define IO_CD_ROM_INCREMENT             1
```
At the end of this post, there will be a link to download all the files needed to build the application and driver. Notice, in the sample sources, that our OnCreate() and OnClose() Dispatch Functions are too similar to the example above. That’s because we don’t take no action when it opens or closes a handle to the device we had created.

Getting parameters from IRP.

In the other OnRead() and OnWrite() Dispatch Functions, we must get data that the driver needs to execute the IRP, such as buffer posted by the user and its size, both in writing and reading. These parameters are into a Stack Location within the IRP. Wow, the more I pray, the more weird names appear to me… Stack Locations structures are parameters that are allocated along with the IRP. There is a Stack Location for each device in the device stack that was called. This conversation can become quite fun, but we have a post to finish. Let’s leave this subject about Stack Locations to our future filter example. There, the matter makes more sense, but if you cannot take such a curiosity and want to know more about it, see what the reference says about Stack Locations. For now, let’s just consider that these parameters are there and to have access to this structure we need to use IoGetCurrentIrpStackLocation() macro. To have a more practical idea of all this bullshit, here it goes all the OnWrite() function code with tons of comments.
```
/****
```
```
***     OnWrite
```
```
**
```
```
**      This routine is called, whenever an application calls WriteFile()
```
```
**      using our device handle as a parameter.
```
```
*/
```
```
 
```
```
NTSTATUS
```
```
OnWrite(IN PDEVICE_OBJECT  pDeviceObj,
```
```
        IN PIRP            pIrp)
```
```
{
```
```
    PIO_STACK_LOCATION  pStack;
```
```
    PVOID               pUserBuffer;
```
```
    ULONG               ulSize;
```
```
    PBUFFER_ENTRY       pBufferEntry = NULL;
```
```
    NTSTATUS            nts;
```
```
    BOOLEAN             bMutexAcquired = FALSE;
```
```
 
```
```
    __try
```
```
    {
```
```
        //-f--> Say hello to the debugger
```
```
        KdPrint(("Writing into EchoDevice...\n"));
```
```
 
```
```
        //-f--> The Buffer address is a parameter that comes
```
```
        //      from the IRP
```
```
        pUserBuffer = (PCHAR)pIrp->AssociatedIrp.SystemBuffer;
```
```
        ASSERT(pUserBuffer != NULL);
```
```
 
```
```
        //-f--> Get the current stack location address from the IRP
```
```
        pStack = IoGetCurrentIrpStackLocation(pIrp);
```
```
 
```
```
        //-f--> Get the buffer size
```
```
        ulSize = pStack->Parameters.Write.Length;
```
```
 
```
```
        //-f--> Here, the node and the string buffer
```
```
        //      are allocated all together
```
```
        pBufferEntry = (PBUFFER_ENTRY) ExAllocatePoolWithTag(
```
```
            PagedPool,
```
```
            sizeof(BUFFER_ENTRY) + ulSize,
```
```
            ECHO_TAG);
```
```
 
```
```
        //-f--> If there is no memory, forget it...
```
```
        if (!pBufferEntry)
```
```
            ExRaiseStatus(STATUS_NO_MEMORY);
```
```
 
```
```
        //-f--> Initializing the structure
```
```
        pBufferEntry->pBuffer = (pBufferEntry + 1);
```
```
        pBufferEntry->ulSize = ulSize;
```
```
 
```
```
        //-f--> Do the copy string from the buffer sent by the user
```
```
        //      to the buffer allocated here.
```
```
        RtlCopyMemory(pBufferEntry->pBuffer,
```
```
                      pUserBuffer,
```
```
                      ulSize);
```
```
 
```
```
        //-f--> Acquire the mutex that protect the list
```
```
        //      against simultaneous accesses.
```
```
        nts = KeWaitForMutexObject(&g_Mutex,
```
```
                                   UserRequest,
```
```
                                   KernelMode,
```
```
                                   FALSE,
```
```
                                   NULL);
```
```
        if (!NT_SUCCESS(nts))
```
```
            ExRaiseStatus(nts);
```
```
 
```
```
        //-f--> We need to remember this in case something
```
```
        //      really bad happens.
```
```
        bMutexAcquired = TRUE;
```
```
 
```
```
        //-f--> Insert the new node to the list's tail.
```
```
        InsertTailList(&g_BufferList,
```
```
                       &pBufferEntry->Entry);
```
```
 
```
```
        //-f--> Tell the IoManager that all data sent
```
```
        //      to the driver were read successfully.
```
```
        pIrp->IoStatus.Information = ulSize;
```
```
        nts = STATUS_SUCCESS;
```
```
    }
```
```
    __except(EXCEPTION_EXECUTE_HANDLER)
```
```
    {
```
```
        //-f--> Get the error code.
```
```
        nts = GetExceptionCode();
```
```
 
```
```
        //-f--> That will force the debugger to stop here.
```
```
        //      But only when compiled in Checked mode.
```
```
        ASSERT(FALSE);
```
```
        KdPrint(("An exception occurred at " __FUNCTION__ "\n"));
```
```
 
```
```
        //-f--> If something gets wrong and we have already allocated
```
```
        //      this buffer, so let's release it.
```
```
        if (pBufferEntry)
```
```
            ExFreePool(pBufferEntry);
```
```
 
```
```
        //-f--> Tell the IoManager the no data were transferred.
```
```
        pIrp->IoStatus.Information = 0;
```
```
    }
```
```
 
```
```
    //-f--> Release the mutex
```
```
    if (bMutexAcquired)
```
```
        KeReleaseMutex(&g_Mutex,
```
```
                       FALSE);
```
```
 
```
```
    //-f--> Complete the IRP.
```
```
    pIrp->IoStatus.Status = nts;
```
```
    IoCompleteRequest(pIrp, IO_NO_INCREMENT);
```
```
    return nts;
```
```
}
```
Another important point to notice here is about filling the Information member on IoStatus structure that is in the IRP. In those data transfer functions, this field informs IoManager the data amount that was transferred from the application to the driver and vice versa. This field directly reflects on the fourth parameter of WriteFile() API, which has exactly the same function. Once received and validated the parameters, we allocate a node that will receive the buffer. Notice that we are allocating in paged memory; after all, all our functions will be executed in PASSIVE_LEVEL. Although this function is a Dispatch Function, it does not mean that it runs in DISPATCH_LEVEL. Take it easy, these are very different things. Noting… “Post about IRQLs and POOL_TYPEs”. The OnRead() function is similar to OnWrite(), so I’ll spare you of putting all code here.

When my driver is unloaded

OnDriverUnload() function will be called when the driver is being unloaded. Here, in addition to empty the buffer list that may have been forgotten in the driver, let’s delete the Symbolic Link and DeviceObject that was created at startup time. Simple as that…
```
/****
```
```
***     OnDriverUnload
```
```
**
```
```
**      The party is over, go home, regards for your wife
```
```
**      and a kiss in your kids.
```
```
*/
```
```
 
```
```
VOID OnDriverUnload(IN PDRIVER_OBJECT   pDriverObj)
```
```
{
```
```
    PLIST_ENTRY     pEntry;
```
```
    PBUFFER_ENTRY   pBufferEntry;
```
```
 
```
```
    //-f--> Say good night
```
```
    KdPrint(("Terminating KernelEcho driver...\n"));
```
```
 
```
```
    //-f--> Here we remove all nodes that weren't read by
```
```
    //      the application. That would happen if the application call
```
```
    //      WriteFile() but not ReadFile().
```
```
    while(!IsListEmpty(&g_BufferList))
```
```
    {
```
```
        //-f--> Take the first list node.
```
```
        pEntry = RemoveHeadList(&g_BufferList);
```
```
 
```
```
        //-f--> Get the outer structure from its node address.
```
```
        pBufferEntry = CONTAINING_RECORD(pEntry, BUFFER_ENTRY, Entry);
```
```
 
```
```
        //-f--> Finally, release the memory used by
```
```
        //      this node.
```
```
        ExFreePool(pBufferEntry);
```
```
    }
```
```
 
```
```
    //-f--> Deleting DeviceObject and SymbolicLink
```
```
    //      created at startup time.
```
```
    IoDeleteSymbolicLink(&g_usSymbolicLink);
```
```
    IoDeleteDevice(pDriverObj->DeviceObject);
```
```
}
```
Wow, that horrible mistake! What if the driver is finished while some reading or writing operation is being performed? Does a dark future await us and our souls will be damned for eternity? Does the Little Mermaid have something to do with it?

Well, better to let our beliefs aside and focus on the DDK. A driver cannot be terminated while there are some references to this device driver. Note that this routine has no return so, we cannot tell the system that the driver may or may not be unloaded. If an application still has an opened handle to any device while you ask to stop it, the system will respond that the driver cannot be terminated. In this condition, the OnDriverUnload() routine will not be called. But otherwise, if nothing prevents the driver from being unloaded and our routine is called, forget it… Your driver is already going to driver’s heaven.

The wonderful world of Userland

I will not put all the application source code in the post, but all sources are on a file available for downloading. I think one thing that’s worth showing here is the syntax of how to get the handle to the device we have created in our sample driver.
```
    //-f--> Here we open a handle to the device that
```
```
    //      was created by the driver. Remember that our
```
```
    //      sample driver must be installed and be working,
```
```
    //      in order the call bellow can work correctly.
```
```
    hDevice = CreateFile("\\\\.\\EchoDevice",
```
```
                         GENERIC_ALL,
```
```
                         0,
```
```
                         NULL,
```
```
                         OPEN_EXISTING,
```
```
                         0,
```
```
                         NULL);
```
Once the handle to the device is obtained, the Read, Write and Close operations will follow exactly as if we were performing the same operations with files. You don’t need to be a Jedi master to be able to use these functions.
```
    //-f--> It sends the received string to the driver via
```
```
    //      WriteFile.
```
```
    if (!WriteFile(hDevice,
```
```
                   szBuffer,
```
```
                   dwBytes,
```
```
                   &dwBytes,
```
```
                   NULL))
```
Installing and testing

I have shown in another post how to install a driver by hand. However, there are more civilized ways to install a driver. One of them is using the OSR Driver Loader, a tool that is offered by OSR to install your driver without rebooting the machine. Actually, this is a very simple procedure to do, but not simple enough to comment about it yet in this post, so let’s use the tool for now.

After compiling the driver, put a copy of it in the System32\drivers of the victim machine. Then run the DriverLoader and fill in the fields as shown in the figure below.

Then click on Register Service to install the new driver and then click on Start Service to start the driver. Okay, now you can use the test application. The application is very simple to use. Once started, enter the strings that should be sent to the driver. An empty string indicates the end of the strings and then it starts reading the same string queued in the driver.

Phew! As we have seen, even a driver that does something simple requires a considerable amount of code and many different concepts. I know that some gaps are remained in the post, but I hope I have helped. If you have questions at some points at driver or even at test application, do not hesitate to ask or send your comments. The contacts are very helpful to define the next posts.
Have fun!

KernelEcho.zip
June 18, 2007
Step into Kernel (VmWare+WinDbg)
It is enough with this idle talk and let’s go to what really matters. Speaking about Kernel Debugging, I have written a post describing the steps needed to Kernel Debugging with WinDbg, using two machines and a serial cable. But having two machines dedicated to this practice is a luxury that not everyone has. Thus, in another post, I have commented about installing and using SoftIce to debug drivers on a “single machine dedicated to debug”. I tell you “dedicated machine to debug” because using your development machine to debug drivers may not be one of your best ideas. The most courageous and confident people in their own code still risk themselves in this practice. Well, I cannot say much, I myself have already done that in lean times. Anyway, we have not fled the need for two machines to have a minimal environment for development and test drivers. An alternative to these scenarios is the use of a single machine, but the one that can have enough memory and CPU to run a virtual machine, in order to debug the kernel. In this post, I will take the steps to use a virtual machine as a guinea pig to test and debug drivers.

Configuring a virtual machine

The natural communication way between real machines using WinDbg to do Kernel Debugging is a serial port. To follow the same steps, we need to make your virtual machine to gain a serial port to make this communication possible. With your virtual machine turned off, click on the Edit vitual machine settings option.

A window will appear with the list of devices already installed on your machine in Hardware tab. Click Add… in order to add a new device and select Serial Port in the device list that is displayed on the screen below.

After clicking on Next, select the Output named pipe. This will cause the serial port on the virtual machine to communicate with the real machine, via a named pipe. Clicking on OK, the specific settings of the named pipe will be presented. The pipe name will be used later in one of the WinDbg settings so, if you want to use a name other than the one suggested here, try to remember this same name later. Then, change the value of the second combo to indicate that the other communication ending will be an application, in this case WinDbg. After that, just click on Finish.

At the end of these settings, select the Yield CPU on poll option as shown below.

Configuring the TARGET machine

Made all that “hocus-pocus”, now we have to set the TARGET machine system in the way it will able to make the Kernel Debugging. Remember that from Windows that runs inside the virtual machine point of view, the named pipe does not exist, it is just a serial port. If you still do not know what TARGET machine is a or how it can be set, then take a look in this post and follow the described steps in the part where a TARGET machine is configured .

Configuring the HOST machine

Assuming that your TARGET machine has been configured, now we have to configure WinDbg in a way it can be connected to a machine using named pipe, instead of a serial port. For that, I usually create a batch file that contains the following command line:
```
start C:\Progra~1\Debugg~1\windbg.exe -b -k com:pipe,port=\\.\pipe\com_1,resets=0
```
Creating a batch file is not a required step, you might want to retype everything in the Run… window, every time the debugging starts; but it’s all up to you. Notice that the pipe name appears here. I hope you still remember which name you have chosen.

Connecting…

Now we have everything already configured, it is just plug and debug. In this story, the virtual machine is the one that creates the named pipe that is opened by WinDbg. Thus, if you start WinDbg with the shown parameters before starting the virtual machine, you will see the window below stating the pipe was not found.

So, the sequence is, firstly, to connect the virtual machine, select the debug option in the Boot as shown below, and only after that, you should start the WinDbg with the parameters described above.

When WinDbg is finally connected to the virtual machine, via the named pipe, we have the following messages in our Command Window within WinDbg.

Thereafter, you know… You pick up the bugs.

Once more I hope I have helped you.
See you next time. 🙂
May 31, 2007
Now he only talks about that…

Registration for Windows Drivers’ course has already started operating this weekend. Because of some delays, the course starting date was postponed and has also been confirmed for June 23.

To see the complete list of extension courses offered by the university, select the item “Information technology” item from “Extension” on the University web site.

For those who enjoy creating Penguins, my hard-coder friend William, will be giving a course about driver development for Linux at the same university. He will also use one of the OSR hardware training kits which appear in the photo above just to give you the opportunity to get your hands dirty. In the case of Linux course, the hardware is the USB one.

See you…

May 21, 2007
Windows Drivers Course
As some of you might read in another post, I was invited by Gama Filho University to give a course about Kernel Mode development for Windows. All the course details have already been determined and they would be producing material for publication; bu, as this post was being written, I was notified that there had been a delay in producing this material and it would be available for May 21. I already took a look in the Folder they were producing, and from what I got, they decided to summarize the course description that would be given. For this reason, I’ll put the full version of the description here. The following was the file I sent to the university.
```
Aim:
```
```
========== 
```
```
This course is intended for developers or students who need to understand
```
```
the fundamental concepts on drivers' implementation for Windows. This course
```
```
would not cover specific drivers' implementation, such as printers, video,
```
```
SCSI, NDIS, USB, 1394 or UMDF. The aim of this course is to prepare students
```
```
who want to understand, test, complement or build drivers for Windows, using
```
```
general concepts involved in the process.
```
```
 
```
```
 
```
```
Pre-requisites:
```
```
==================
```
```
Knowledge in C Language 
```
```
Windows API Basics
```
```
Operating Systems Basics
```
```
 
```
```
 
```
```
Covered Topics:
```
```
=====================
```
```
System Architecture Overview
```
```
        Processes and Threads     
```
```
        Virtual Memory and Paging
```
```
        Kernel Mode x User Mode
```
```
        Subsystem and Native API
```
```
        IoManager
```
```
        Driver Stack and Plug-and-Play
```
```
        Object Manager
```
```
                Terminal Server
```
```
        Hardware Abstraction Layer (HAL)
```
```
 
```
```
Environment (Getting, installing and using)
```
```
        Windows Device Driver Kit
```
```
        Microsoft Visual Studio Express
```
```
        Microsoft Windows Debugging Tools
```
```
        Symbols
```
```
 
```
```
Writing a Driver
```
```
        Writing DriverEntry and DriverUnload
```
```
        Compiling a Driver
```
```
        Installing a Driver (Legacy)
```
```
                Dependencies
```
```
                Groups
```
```
                Load Order
```
```
        Debugging a Driver
```
```
                Checked Build Installations
```
```
                Driver Verifier
```
```
                Mapping an image for debugging
```
```
                Using Virtual Machines
```
```
                SoftIce
```
```
        Creating DeviceObject
```
```
        Symbolic Links
```
```
        I/O Request Packets
```
```
        IOCTLs and DeviceIoControl
```
```
        Implementing Dispatch Routines
```
```
                Buffered I/O
```
```
                Direct I/O
```
```
                Neither I/O
```
```
        Objects, Handles and Pointers
```
```
        Arbitrary Context
```
```
        IRQLs, APCs, DPCs and WorkItems
```
```
        Synchronism
```
```
                Mutex
```
```
                FastMutex
```
```
                ERESOURCE
```
```
                Spin Lock
```
```
        Events and Timers
```
```
        Custom Queues
```
```
 
```
```
Hardware Interaction
```
```
        Port I/O
```
```
        Interruptions and ISRs
```
```
        DMA
```
```
 
```
```
Writing Filters
```
```
        Writing AddDevice routine
```
```
        Legacy Filter Drivers
```
```
        Forwarding IRPs
```
```
        Stack Locations
```
```
        Completion Routines
```
```
        Pending IRPs Handling
```
```
        IRPs Cancelation
```
```
        Creating IRPs for third Drivers
```
```
 
```
```
Drivers Types
```
```
        Legacy drivers
```
```
        WDM Drivers
```
```
        Minidrivers
```
```
        Miniports
```
```
        Miniclass
```
```
 
```
```
Installations
```
```
        Creating an .INF file
```
```
        The use of SetupApi
```
```
 
```
```
References
```
```
        Web Sites
```
```
        Discussion Lists
```
```
        Books
```
```
 
```
By the time this post was written, the university attendance was not able to provide details about the course, such as registration dates, contents or due date. But I can anticipate what I know so far.

The course will be 40 hour-long and it is divided into 10 classes of four hours each. The classes will be on Saturdays from 1:00pm to 5:00pm.

The course is scheduled to have its first class starting May 26. It will necessary, at least, five students to form one class, but by taking the amount of people who have already contacted the university seeking for details (without disclosure) my concern is now the upper limit of 10 students. We decided to limit the class of 10 students in order there would be a better time usage for the content.

I agree that 40 hours is not enough to learn everything you need to develop drivers, but it will be an excellent starting point to have the first contact. If you are the type of person who needs to learn absolutely everything to start developing then I can antecipate that this course is not for you. The best kernel developers I know (from their Blogs, but I know) have been working with drivers for years and years, but they say they have never know everything. To get an idea, some developers focus on just certain issues within the kernel. One has been working only with disk drivers for about 10 years; another one, for about seven years, only with network drivers. Tony Mason, for example, has been working for 18 years, only with File System Drivers. Now, ask them if they know everything! And I used to feel myself bad for trying to focus only on working with Kernel, refusing to work in User Mode. Sometimes, with friends, I have heard comments about how to do this or that using .Net, and I, having no idea what they were talking about. I didn’t know that inside the Kernel issue have still existed a myriad of details and specialization. Is it possible to learn everything?

I am preparing the content in a way that students may have some practical experience, such as writing and compiling a “Hello World” driver, install them in virtual machines and debug them. There is no way. It is necessary to get your hands dirty. Connect two real machines with a serial cable and do Kernel Debug. Generate and analyze Crash Dumps. I think we have to make the most possible to do things together in one classroom, and perform some experiments that the books attempt to describe only in words and pictures. This desire of putting things in practice was the responsible for contributing with me to get the training kit on sale at OSR Online. The kit that I’ll be taking to the course, and that appears in the photo above is basically a PCI Digital I/O. My goal is to make students’ drivers control this device.

Well, as soon as I’m getting news, I will let you know.
See you…
May 10, 2007
How was it in Boston?

As I commented in my last post, I attended the seminar for Windows File System driver development produced by OSR in Boston. In this post, I will talk a bit about how things went there.

To begin with, none other than Tony Mason to give this seminar that was very, very helpful. For those who are not familiar, Tony Mason is one of the four OSR partners. He has been working with drivers over 20 years, and particularly with File System drivers since, 1989. He began working with File Systems for Windows NT in 1993, but was already developing for Unix File Systems before that. Tony Mason is also the person responsible for reviewing and updating Windows NT File System Internals book contents by Rajeev Nagar, written and published in 1997. This book is now a reference in the subject and its revised edition, which still has no release date, will bring new issues that have been implemented in Windows 2000, XP and Vista. I have also commented a bit about this in this post.

You people can call me a “super fan”, but it is nice to see how it just takes implementation details of File Systems Drivers (FSD) and File Systems Filters (FSF). I obviously asked him to sign up my book copy. Now I can keep that autograph on my calendar next to my first absorbent packaging. I’d love to show here, but it is about his signature, I don’t think it would be a good idea to put it on the Internet.

We were fifteen students in the class, where twelve were Americans, one Romanian, one Canadian and one Brazilian. I had the opportunity to be the first Brazilian to take this course from OSR. There were people of various ages. The youngest was about 25 years-old, whereas the oldest had his hair completely white. I imagine that the class average age was about to 40-years old. I’m comfortable with the certainty that, in this development area, there is no risk of being replaced by kids who just got out of college, just by lower prices of labor. Tell me the truth! How many 40 years-old programmers do you know? I said programmers, not managers. Well, fortunately the driver development will still require a lot of experience.

It’s hard to believe, but a woman (about 43 years-old and really looking like a woman!) was also part of the group. It is not sexism. You know how it is unusual to see a woman (who does not look like a man) programming, but programming kernel it is even more unusual. Another specimen of this rare species is Molly Brown, a programmer responsible for the NT Cache Manager. Check here.

As it might be expected, the seminar rhythm was accelerated. After all, the content was quite dense. We had an overview of IoManager, virtual memory, multi-threaded, multi-processing and Object Manager just to begin with. These issues are still operating system part only and not specifically for FSD. We started to see File System contents itself at the middle of the second day. All content had been well distributed and for each point explained, we received comments focusing on the FSD, FSF and Redirectors. Interesting to see how each item was discussed, getting comments on the implementation differences among different Windows versions.

In our group, there was at least one person who would develop a Redirector, one that would develop a File System, but the public in general would improve their knowledge to develop anti-virus, anti-malware, access control to systems and data replication. All implemented as FSF.

Data replication? Would not it be better to develop data replication with a disk filter? Yeah, I think so either. The File System interface is much more complex than the disk interfaces, but try to explain that to that guy! My sincere wishes of good luck to him.

I can say that the student’s level was medium. Well, I’ve read the book from Rajeev a few times and I already had the opportunity to work with FSF. I was aware of I would not waste this opportunity to be there learning things that a book could teach me, but part of the group didn’t think the same. Maybe they were thinking it would be easy. It was funny when we had our coffeebreaks and they sed to commente that the matter was very thorough and very detailed. No doubt it was! Some of them never had contact with FSD before. I can say who already had contact with the subject and was able to enjoy the details of interaction between the FSD and other system components such as the Cache Manager, Memory Manager or even the LAN Manager Server, really enjoyned the course. There were some people who got stuck on comments about IoMarkIrpPending and Completion Routines. Seriously? I’m saying this issue is trivial, but you should not leave to learn this in a FSD seminar.

There were very good people there, too. Questions and discussions made Tony open FastFAT, CDFS or RDR sources to show how things would be implemented in the real world. And I thought that just the reference considered source samples as part of the documentation. Have you already searched for some information about IRP_MN_MDL in reference? Well, I was able to find a clue in the part that talk that IRP_MJ_READ was mentioned, where they listed this, among the other options and said:

“For more information about handling this IRP, study the CDFS and FASTFAT samples that are included in the Windows Driver Kit (WDK).”

According to Tony, developing an FSD requires much knowledge and experience, but unlikely what many people could think, developing filter is even worse. All begin with the false impression that it is necessary to change only a small part of the filter sample from WDK (SFilter), that everything would be okay. But a side effect will require just one more change here, another there, and when you realize we have a complexity greater than a FSD project. A strong argument he used was that, when you implement an FSD, you know everything what may happen, but when you develop a filter, you are at the interface of two black boxes, the operating system and FSD; and any change in behavior, generated by your filter, can generate the most bizarre, intriguing and unexpected situations to be debugged in both black boxes.

The OSR gave us a binder with all PPTs used in the seminar; also, it gave us an OSR portmanteau, a NT Insider edition and to some people’s surprise a crazy spring. A spring crazy? Well, in summary it is a spring, but a crazy one.

If you want to know more about File System Drivers, an excellent source of information is the OSR mailing lists. It’s worth takeing a look. Obviously, I will comment more on this subject here.

See you next time! 🙂

May 2, 2007
Who said it’d be easy?

That’s it boys and girls; time seems to become increasingly scarce. The intermediate exam period of my fourth year in Computer Engineering makes time disappear. As it wasn’t enough, I still insisted on taking some extra work to do at home. What could I do? Someone has to do the dirty work. As some of you have already known, in my university exam period, my posts were very poor and I have even commented on things that do not have much (or almost nothing) to do with driver development for Windows, just like my last Off Topic post. The funny thing is that this post which talks about my adventures programming a HP 50G calculator is one of the most visited pages of this blog. What it is not surprising. Tell me, how many Brazilians do you know who program drivers for Windows? If you could use all the fingers of one hand, congratulations; introduce me to some of your friends. I have worked with this for years and I can barely fill one hand up (counting the people who no longer work with this).

In this post, I will bring a preview about what is coming around for future posts. My exam week will end next Friday, and after that, I imagine myself being able to breathe a little and also being able of torment your lives with this issue that nobody cares about.

But hardware that matters…

From the posts written so far, I still did not tell anything about writing drivers that actually do some manipulation with hardware. That’s right, interruptions, port I/O and so on. Even that Brazil is not a such big hardware producer. In companies that I have worked so far, only during the first one I had to control hardware that maintained data collector network. Among the remaining companies, I used only Kernel Mode privileges to complement some security solution. File System filters for access control, network filters, native APIs hooks, keyboard and mouse filters, disc filters, real-time encryption and blah blah blah… And about hardware? Perhaps the biggest obstacle on developing a device driver is the need to actually have a hardware to control. For those who are unaware, OSR sells training kits addressing to this need. These kits bring boards, circuits, wires, cables, chips … Uhuuu! Finally, hardware. For those who are interested in it, take a look at OSR Online Store at Hardware Learning session that will initially offer two kits. Well, at least, had offered. While I have being writing this post, one of the kits were being removed from the site. One kit is a PCI card with Digital I/O and the other is a USB card. Last week, finally the training kits that I have ordered has arrived and based on them, I have intended to build some basic examples to share with you.

Drivers for Windows course at Sao Paulo

The main objective of these training kits is to provide a practical example in an introductory course about Kernel Mode development for Windows at a university here in Sao Paulo. Last month, I was invited by the university to teach a course on this subject, so little explored in Brazil (I have not found another dedicated site about driver development for Windows in Portuguese). The course has still been prepared and there not a starting date. Beyond what’s been expected from such a kind of course that is usually a course in which a talkative person is supported by tons of PPTs, I intend to provide a relevant differential in this course. I want to give the opportunity for pupils learning enough to be able to develop their own driver, which will control a real board. You know, a course with so much content without the practical part would be like learning about nuclear physics, where we see tons of theory and sometimes we cannot imagine how that would be applied in real life. The examples and PPTs that are created for the course will be available here. Obviously there will be situations and questions that will deserve interesting posts here.

File System Training at OSR

And speaking about courses, that’s my turn as arrived. This Saturday, April 14th, I’ll be going to Boston to attend the File System Development for Windows seminar at OSR. There will be four days with tons of theory on the subject. I intend to give an overview about how the training is organized and how the content rhythm is presented. After all, this subject is not that easy.

Computer Engineering

That last but not least, it is the influence of what I’ve been seeing at the university reflected here. I have mentioned this issue to friends and I see that is inevitable. The study of VHDL, microelectronics, microcontrollers and processors will end up manifested here in the form of Off-Topic posts. Perhaps the issue is not completely off-topic, since low-level software is a matter that is closely tied to hardware. Anyway, I promise to control myself.

Well, after so much, I think it’s easier for you to understand my lack of time being present here. But don’t worry DDK disciples; this blog has brought me so much cool things and I do not intend to stop publishing them suddenly. See you next time…

April 9, 2007
Synchronism x Performance
No wonder that my last posts brought issues related to linked lists and performance. In recent weeks, I had been hired by a security company to take a look at one of their File System filters, in order to reduce the delay caused by them. In this post, I will talk about synchronism and CPU contention.

I think it should not be new to many here that a linked list, or any other resource, when shared among multiple threads, should implement some access synchronism control thus, avoiding a thread to read invalid data as a result of a change made by another thread at same time.

But I don’t have two processors, synchronism for what?

It is true that computers with only one processor run only one thread at any given moment. So we never have two threads being executed at the same time. But it is important to remember that threads are executed in small slices of time determined by the scheduler, considering each thread quantum size and its priority. There are ways of avoiding that a thread be interrupted by the Windows scheduler, but at normal temperature and pressure conditions, we do not know when a thread is interrupted giving its place to another thread be executed.

Suppose that thread A is scanning a list in search of a certain node. This thread reaches record R and is interrupted so that thread B can be executed. Thread B removes that same record R from the list and deallocates it from RAM. When thread A is resumed and reads the fields of the late record R, it will access invalid data and make the following steps unpredictable. Even though it is quite predictable that something blue should happen.

There are several synchronization mechanisms that we can use, but in this post, I will comment specifically about those I’ve been involved in these weeks. But besides these ones, I have to talk about the most exotic I’ve seen in years of experience, which for me was nicknamed “Mutex, pero no Mutcho.” This was a derived class from VMutex of VToolsD. The class enter() method insanely tried to acquire a mutex few thousand times in a loop. If after these thousands of interactions the mutex has not been acquired, then the thread gave up and accessed the shared resource anyway. Do you think the system was having some trouble of Dead Lock? Anyway, it was nice to fix this years ago. There are things we only believe when we see them.

As I have mentioned in another post, File System Filter is a layer placed over drivers like FastFAT, NTFS, CDFS, Network Redirectors and any other drivers that implement a file system interface. That is, all operations related to files, including File Mapping access, pass through File System filters.

If you’re not only interested to know about File System filters, but interested in working with File Systems itself, then you have an obligation to read the Windows NT File System Internals by Rajeev Nagar. This book is the single known reference that is comprehensive enough on this subject. First published in 1997 by O’Reilly, this book is still a mandatory tool for development relative to File Systems even for Windows Vista. Its publication was interrupted a few years ago, and during that time, I saw this book being sold for more than U$ 200.00 for a single used copy on Amazon. Today OSR holds the book copyrights and it is currently working on an updated edition, which should bring issues such as Shadow Copy, Transactional NTFS, Filter Manager, Mini Redirectors and force volume dismount. However this is a job that will take some time, then in 2005, the original version was reprinted to meet this need, while the new edition has been made.

The filter I have worked on maintains several lists to be consulted on each intercepted access. The mechanism used to synchronize access to these lists was the Spin Lock. An obvious choice, but inadequate for this scenario, where the activity is too intense. The list group that is used on IRP_MJ_READ routine is also used at IRP_MJ_WRITE routine and other events. The result is CPU contention. That is, when a thread gets a Spin Lock to do a query on the list, all other threads that need to consult the same list have to sit and wait until the Spin Lock is released (even in cases that we have more than one processor). Knowing that these lists are not small. Can you imagine how slow it was?

Just as world hunger, cervix cancer, the prostate examination and other ills that affect mankind, something should be done about this. Therefore, in a relentless struggle against the evil forces was created the ERESOURCE. (Angelic chants and a dry ice mist is dissipated into the ground)

Using ERESOURCE

ERESOURCE is the most appropriate and native way used to synchronize access to structures that are part of a file system, such as FCB (File Control Block), which besides other information, keeps the current file size. With ERESOURCE, multiple threads can access the same linked list at the same time for reading. Assuming that none of the threads would change the data list, then all accesses could be performed simultaneously. In contrast, if necessary, a thread can gain exclusive access to the list in order to make a change.

To use ERESOURSE, you must declare a ERESOURCE variable, which must reside in non-paged memory and 8 bytes aligned, and initialize it using ExInitializeResourceLite() function. For shared access (read only) to the list, you must use ExAcquireResourceSharedLite() function. This function checks for a thread with exclusive access to controlled resource. If so, the function may, depending on a parameter, return failure or wait until the resource to be released. To have exclusive access, use ExAcquireResourceExclusiveLite() function, which analogously checks for threads with shared access to the resource, and optionally, waits until all threads with shared access before releasing the resource for exclusive access. Finally, to release the access granted, either exclusive or shared, use the ExReleaseResourceLite() function.

An interesting point to notice is that the Kernel APC delivery must be disabled for the threads that acquire ERESOURCE. I do not know the real reason for this need, but at least it prevents the thread holding the resource from being terminated by another thread, knowing that the TerminateThread() is implemented via Kernel APC. Thus, the most common way to use these functions is as shown below.
```
    //-f--> Get shared access (read only).
```
```
    KeEnterCriticalRegion();
```
```
    ExAcquireResourceSharedLite(&m_Resource, TRUE);
```
```
 
```
```
    //-f--> Query the protected values.
```
```
 
```
```
    //-f--> Release the resource.
```
```
    ExReleaseResourceLite(&m_Resource);
```
```
    KeLeaveCriticalRegion();
```
One downside is that most of the functions that deal with ERESOURCE, unlike Spin Locks, should be called with IRQL < DISPATCH_LEVEL. Thus, perhaps some tricks are necessary when handling IRPs and their CompletionsRoutines to deal with this.

After those modifications, the system has gained a lot of performance on the tests I did. The gain will be even greater as we have more operations in parallel, and of course, on computers with more than one processor.

And everybody has lived happily ever after…
February 28, 2007
Linked lists on DDK
Since the beginning of my studies, I have always chosen to study something that would unite computer science with electronics, and this led me to choose the Computing Industrial course at ETE Jorge Street. An excellent school and I learned a lot there. However I have only learned what a linked list is in a professional environment during my internship. Years later, the university introduced me to these lists in form of classes written in Java, where we dealt with references and garbage collectors. Even those who work with Visual C/C++, handles, templates and classes offered by MFC, ATL, WTL and STL, which end up abstracting real linked list implementation. In this post, I will talk a little about resources offered by DDK regarding to this subject.

But I am already a big boy and I already know how to build linked lists in C/C++. Why should I use DDK resources?

If you only store private data, such as a list of buffers to be sent to the device, that’s fine, but to deal with DDK structures, it would be, at least, interesting to know how they are stored in lists. Some situations require you to know how to use DDK lists. An example is: If you implement the custom control queue for IRPs in your driver, the pIrp->Tail.Overlay.ListEntry field will be free to be used while such IRP is held by your driver.

The simplest of these resources is SINGLE_LIST_ENTRY structure that is defined at ntdef.h, as shown below:
```
typedef struct _SINGLE_LIST_ENTRY {
```
```
  struct _SINGLE_LIST_ENTRY *Next;
```
```
} SINGLE_LIST_ENTRY, *PSINGLE_LIST_ENTRY;
```
A SINGLE_LIST_ENTRY variable must be set to be the head of our list. This variable should have its Next member initialized with NULL before being used. The PushEntryList() and PopEntryList() routines are used respectively to add and remove elements from the list head. Like most linked lists, nodes are referenced by the Next member until it is NULL, thus indicating the node chain ending.
```
VOID 
```
```
  PushEntryList(
```
```
    IN PSINGLE_LIST_ENTRY  ListHead,
```
```
    IN PSINGLE_LIST_ENTRY  Entry
```
```
    );
```
```
 
```
```
PSINGLE_LIST_ENTRY 
```
```
  PopEntryList(
```
```
    IN PSINGLE_LIST_ENTRY  ListHead
```
```
    );
```
But wait a minute. All this is beautiful, but isn’t anything missing? Like any linked list structure, among the members of each node, there must be one which points to a similar structure, which is the link to the list next node. In the example below, the pNext member is the one responsible for it.
```
//-f--> Structure that defines the node list
```
```
//      which will be used as an example.
```
```
typedef struct _MY_NODE
```
```
{
```
```
    //-f--> Your private data that will be defined
```
```
    //      by yourself.
```
```
    UNICODE_STRING      usSomeData;
```
```
    ULONG               ulAnotherData;
```
```
    struct _MY_NODE     *pNext;
```
```
    PVOID               pMoreData;
```
```
 
```
```
} MY_NODE, *PMY_NODE;
```
But from what we could see at the DDK structure, there are no useful members on it, those ones that contain information that will be kept in the list, like fields as usSomeData or pMoreData. At the structure offered by the DDK, there is only the next node address.

What is the point on saving only nodes?

Actually, the SINGLE_LIST_ENTRY structure as well as other DDK lists structures should be used in conjunction with the CONTAINING_RECORD macro. This macro returns the structure base address that contains a known field in a known address. Well, I have tried to rewrite this sentence three times, but I think you will only understand when you see the example below. Suppose we are using the SINGLE_LIST_ENTRY in our previous example. Thus, we would have the following structure:
```
//-f--> Structure that defines a register on our linked list
```
```
//      and will be used in this example
```
```
typedef struct _MY_NODE
```
```
{
```
```
    //-f--> Register's private data on the list node.
```
```
    //      These fields are not fixed and defined by you.
```
```
    UNICODE_STRING      usSomeData;
```
```
    ULONG               ulAnotherData;
```
```
 
```
```
    //-f--> That member doesn't necessarily need
```
```
    //      to be the first or the last one at this structure;
```
```
    //      it can be at any position here inside.
```
```
    SINGLE_LIST_ENTRY   Entry;
```
```
 
```
```
    //-f--> More data
```
```
    PVOID               pMoreData;
```
```
 
```
```
} MY_NODE, *PMY_NODE;
```
Our structure is basically the same, except the member pNext, which has now been changed to use the structure SINGLE_LIST_ENTRY. Notice that the field doesn’t need to be either at the beginning or the end of our structure. The example below demonstrates how to include nodes in lists composed by SINGLE_LIST_ENTRY nodes.
```
NTSTATUS PrepareDataAndPush(VOID)
```
```
{
```
```
    PMY_NODE            pMyNode;
```
```
 
```
```
    //-f--> Here we get the already allocated node
```
```
    //      with its fields correctly initialized.
```
```
    pMyNode = AllocateAndFillNode();
```
```
 
```
```
    //-f--> Testing is never too much.
```
```
    ASSERT(pMyNode != NULL);
```
```
 
```
```
    //-f--> To put the node at the list, it is necessary to use
```
```
    //      the SINGLE_LIST_ENTRY member address.
```
```
    PushEntryList(&m_ListHead, &pMyNode->Entry);
```
```
 
```
```
    return STATUS_SUCCESS;
```
```
}
```
At the time to include the node in the list, we use the SINGLE_LIST_ENTRY member address, as suggested by the PushEntryList prototype function. Follow the example below that demonstrates how to get this node using PopEntryList function. This function returns the address passed to PushEntryList function, which is a PSINGLE_LIST_ENTRY. From this address, as I had commented before, we can get the base address of our structure using CONTAINING_RECORD macro. See the example below.
```
NTSTATUS PopAndProcessNode(VOID)
```
```
{
```
```
    PSINGLE_LIST_ENTRY  pEntry;
```
```
    PMY_NODE            pMyNode;
```
```
 
```
```
    //-f--> Here we get the node from the list.
```
```
    pEntry = PopEntryList(&m_ListHead);
```
```
 
```
```
    //-f--> Now we can check if a register really was retrieved
```
```
    //      from the list.
```
```
    if (!pEntry)
```
```
        return STATUS_NO_MORE_ENTRIES;
```
```
 
```
```
    //-f--> Now we use the CONTAINING_RECORD macro
```
```
    //      to get the base address we need.
```
```
    pMyNode = (PMY_NODE) CONTAINING_RECORD(pEntry, MY_NODE, Entry);
```
```
 
```
```
    //-f--> Now we can access the whole structure.
```
```
    ProcessAndFreeNode(pMyNode);
```
```
 
```
```
    return STATUS_SUCCESS;
```
```
}
```
This same style is also used with the LIST_ENTRY structure, which is the one used in most drivers. This structure allows you to create doubly linked lists, and like SINGLE_LIST_ENTRY, a LIST_ENTRY variable is set to be our list head. Its initialization is done using InitializeListHead() function, which initializes the Flink and Blink members with the list head address.
```
typedef struct _LIST_ENTRY {
```
```
  struct _LIST_ENTRY *Flink;
```
```
  struct _LIST_ENTRY *Blink;
```
```
} LIST_ENTRY, *PLIST_ENTRY;
```
The Flink member points to the next node, whereas the Blink member points to the previous node. Unlike most linked lists we know, its endings are not marked by NULL at Flink and Blink members. Instead, their ending points to the address of the node designated as our list head. The example below demonstrates how we could sweep this kind of list, searching a certain node. Before performing any operation with the lists composed by LIST_ENTRY structures, use IsListEmpty() function.
```
//-f--> Looks for a node through a list
```
```
//      composed by nodes using LIST_ENTRY structures.
```
```
BOOLEAN SearchEntry(PUNICODE_STRING  pusLookFor)
```
```
{
```
```
    PLIST_ENTRY pEntry;
```
```
    PMY_NODE    pMyNode;
```
```
 
```
```
    //-f--> Before any operation, it is necessary
```
```
    //      to check for an empty list condition.
```
```
    if (IsListEmpty(&m_ListHead))
```
```
        return FALSE;
```
```
 
```
```
    //-f--> Get the first node address.
```
```
    pEntry = m_ListHead.Flink;
```
```
 
```
```
    //-f--> While pEntry is not the head list node address,
```
```
    //      it means we have a valid node address.
```
```
    while(pEntry != &m_ListHead)
```
```
    {
```
```
        //-f--> Get the node structure base address.
```
```
        pMyNode = (PMY_NODE) CONTAINING_RECORD(pEntry, MY_NODE, Entry);
```
```
 
```
```
        //-f--> Check if this is the node we're looking for.
```
```
        if (RtlEqualUnicodeString(pusLookFor,
```
```
                                  &pMyNode->usSomeData,
```
```
                                  TRUE))
```
```
        {
```
```
            //-f--> Returning TRUE to indicate the node was found.
```
```
            return TRUE;
```
```
        }
```
```
 
```
```
        //-f--> Get the next node address.
```
```
        pEntry = pEntry->Flink;
```
```
    }
```
```
 
```
```
    //-f--> If we got here, it means that the node we were looking for
```
```
    //      was not found. So, returning FALSE to indicate that.
```
```
    return FALSE;
```
```
}
```
NOTE: The above example is only used to illustrate the search for a node. To implement a search like this in a production environment, you must keep in mind issues about multiple thread access synchronism control and reference counters.

To manipulate lists formed by LIST_ENTRY, you can use routines like InsertHeadList(), InsertTailList(), RemoveHeadList() and so on. I will not list all the functions here, I’m sure DDK reference do it better than me.

For both types of lists, there are functions like ExInterlockedXxxList that take a Spin Lock to synchronize changes in the list. Remember that if you synchronize access to lists using Spin Locks, all nodes must reside in non-paged memory. As some of you may know, when acquiring a Spin Lock, the thread IRQL goes to DISPATCH_LEVEL. In this IRQL, Memory Manager is unable to retrieve data page that was paged out to disk, resulting in a BugCheck.

It is also important to remember that you should not mix the use of ExInterlockedXxxList functions with unsynchronized ones. If a list is accessed by multiple threads, all access to it should be controlled.
```
typedef struct _SLIST_ENTRY {
```
```
  struct _SLIST_ENTRY *Next;
```
```
} SLIST_ENTRY, *PSLIST_ENTRY;
```
The SLIST_ENTRY structure is an alternative to SINGLE_LIST_ENTRY lists with synchronized access. This version aims at being more efficient using the functions like ExInterlockedPopEntryList() and ExInterlockedPushEntryList(). Unlike other structures presented here, the list ending is formed by a different structure of that used on its nodes; this structure is the SLIST_HEADER, which is an opaque structure that should be initialized by ExInitializeSListHead() function.

Other advantage on using this list type is that DDK offers the ExQueryDepthSList() function, which returns the number of nodes stored the list.

If your main problem is performance, then the ideal situation would be to use the group of functions that work with Generic Tables. The list head is defined by an opaque structure called RTL_GENERIC_TABLE. This structure, along with its manipulation functions, such as RtlGetElementGenericTable(), creates binary trees with self balancing. What a chic thing, huh?

But Generic Tables is an issue that deserves a dedicated post. Until next time!
February 24, 2007
Cool, but what is an IRP?

My friend Slug, has just read the title of my last post, he laughed and said that most people who read this would ask, “What is an IRP?” Well, thinking about what he said and taking into consideration the a non-simplified explanation DDK reference offers us, I will write a superficial description about IRP, rid my conscience of this weight and be able to sleep again.

Let’s leave the frightening and horrifying DDK diagrams aside to try to see things in a somewhat simpler way. After that, you may refer to the sacred documentation to reinforce the concepts and take any questions about something you have already known what it is, or at least imagined.

Let’s rely on a very practical example. The File Systems drivers for example, NTFS and FAT are implemented as kernel drivers which are called File System Drivers. I imagine that many of you have had the opportunity to open and write to a file.

To start talking to a driver, which we initially make a connection with it, that is done using the CreateFile() function. Contrary to it seems, this function is not restricted to file creation, indeed, the fact of opening a file is a specific way to open the connection to the File System driver (Details about this the next time). The CreateFile function will return to us a handle which will be used to interact with the driver through functions such as ReadFile(), WriteFile() and DeviceIoControl(). By the way, they are not restricted to file operations. When an application calls ReadFile() function, the Win32 subsystem forwards this request to NtReadFile() native API, which in turn, makes the transition to Kernel-Mode and invokes the IoManager, which also in turn, will marshal the request parameters in a structure called IRP (I/O Request Packet).

IoManager sends this packet to the responsible driver, passing through the filters that might be installed on it. An anti-virus driver is a perfect example about the scenario we’re talking about. Anti-virus drivers are implemented as File System Filters. When some application writes to a file, the write IRP gets into the anti-virus before it reaches the File System driver, giving it the opportunity that it needs to check if what is being written contains any known virus signature. For a reading request, the IRP passes through the anti-virus, which installs a CompletionRoutine on it, and thus, it gains access to the IRP’s data when the reading is finished by the target driver.

The IRP is basically divided into two parts, the Header and the Stack Locations. In the IRP Header there is general information, such as status, a pointer to the thread to which the IRP belongs to, user’s buffer addresses, this IRP’s cancellation routine address and so on. In the Stack Locations there are specific request parameters. In case of a file reading, there will be the offset, size and buffer alignment.

Several Stack Locations might exist within an IRP. One for each device belonging to the chain of layers that follow until the IRP reaches the target driver. In other words, there is a Stack Location for the target driver and an additional one for each filter driver installed on it. As the IRP goes down the layers, each driver passes the received parameters from its Stack Location to the one belonging to the next driver using theIoCopyCurrentIrpStackLocationToNext() routine. Once copied, the filter can make the desired changes. If there are no changes being made in the parameters from one layer to another, then the IoSkipCurrentIrpStackLocation() routine you can be used.

The IRPs are going from one layer to another when the filter that received it forwards it to the device in which it is attached, using the IoCallDriver() function. When the IRP gets into the target driver, the driver has basically two options. If the request can be answered immediately, the driver does the desired action and completes the IRP using IoCompleteRequest() routine. But if drivers need to communicate to a device, or even to another driver, the IRP is placed in a queue, it is marked as pending and it will only be finalized when all processing is done.

Returning to our example, when a File System driver receives an writing IRP and depending on many other circumstances, the driver sets the current IRP as pending and creates a new IRP for a volume device representing the partitions, which in turn passes the requests to storage devices. Thus, it is easy to understand that disk drivers know nothing about NTFS or FAT. Storage drivers may have additional storage filters, such as a RAID for disk mirroring among other things. These new IRPs are also created by the IoManager and the whole cycle is redone for each new request.

Watching IRPs

IrpTracker is a tool capable of monitoring IRP’s activities for a particular driver or device. See the figure below where I’m selecting all the devices from Kbdclass driver, which are responsible for reading the keyboard. Notice that my machine has many keyboards. That’s because I’ve been working on anti-key-logger solutions. Selecting all these devices will be easy seeing the IRP’s activities to get the keys you press on the keyboard.

For each key you hit on the keyboard, four lines are issued on the IrpTracker. Each strike key represents two movements, the descent of the key and its release. Each movement is taken with an IRP, which comes with its Completion that means, a line is identified as Call and the other as Comp, representing, respectively, the going IRP and its return. Notice that in this example, the line Comp always comes before the line Call.

Does this mean that the IRP had its completion before being sent to the keyboard driver?

Indeed, the IRP responsible for reading the keyboard gets pended until a key is received. Thus, the system sends an IRP to the driver which waits keyboard events. When this event occurs, the driver receives the key, completes the IRP and the system receives its completion. Soon after, the system launches a new IRP to wait for the next event. Thus, we will always see pair lines, these being the previous Comp IRP and the Call for the next IRP.

If you double click one of these lines, you can see the details of each field in the IRP as shown in the figure below.

I am preparing an evolution driver of Useless.sys, which was used as a starting point in a previous post. The changes will allow it not to be so Useless and be able to demonstrate the IRPs handling in practice, but it will be for another time.

See you there…

February 12, 2007

Who owns this IRP? (Process ID)

There are cases that you need to know which process have launched a given IRP. This is very common in Firewalls or other security programs, which intercept I/O operations to check in their databases whether a certain process has or not access to a particular resource or service. But how can I know which process an IRP belongs to?

Well, I can start thinking it’s very easy to do. As we know, IoManager gives us the IRPs in the process context that made the request. Thus, knowing the API PsGetCurrentProcessId() we can get the process ID that launched the IRP. See how simple it is:

/****

***     OnDispatchProc

**

**      A generic name that tells absolutely nothing...

*/

NTSTATUS OnDispatchProc(PDEVICE_OBJECT pDeviceObject,

                        PIRP           pIrp)

    //-f--> I'm thinking it is easy, don't copy.

    HANDLE hProcessID;

    //-f--> Get the current process ID.

    hProcessID = PsGetCurrentProcessId();

...

Can you see how simple it is to think that everything is right and even being mistaken?

In fact, IoManager delivers the IRPs in the process context of which is doing the I/O. However, what if a third filter got attached to your driver? Yes, it is possible, and when these drivers receive these IRPs, it is not guaranteed that they will transfer them to our driver at the same process context. Suppose we write a driver for a device:

An application asks a write operation to the device.
IoManager creates and sends the IRP to our driver.
A filter attached to our device receives the IRP.
The filter performs an asynchronous task, possibly using ExQueueWorkItem().
While this task has not being ended up, the filter will mark the IRP as pending and return STATUS_PENDING.
The asynchronous operation, in this case, is performed by a system thread and at the end of the task, the filter forwards the IRP to the driver below it that in case, it is our driver.
Our driver then receives the IRP in the system context and not in the process context that originated the IRP.

When the filter keeps the IRP pending and returns from the Dispatch function, the thread that originated the IRP goes ahead and will perform other tasks. The original thread can still return to the process that started the whole operation in case the OVERLAPPED structures were used when calling the driver.

The IRP will now be executed in the process context that made a call to IoCallDriver passing the IRP that was pending as a parameter. In our example, the process that will do this is the System process. Using the code above, we obtain the System PID in place of the PID of the process that actually initiated the IRP.

To properly obtain the information we’re looking for, we’ll have to stroll a bit bigger. When every IRP is created, it enters the pending IRP list of the thread that created it. To get this thread from the IRP pointer, we use the field pIrp->Tail.Overlay.Thread. This field has a pointer to the ETHREAD structure, which refers to the thread that created this IRP. To get the process from the thread, we can use the IoThreadToProcess() API. See the excerpt below.

/****

***     OnDispatchProc

**

**      A generic name that tells absolutely nothing...

*/

NTSTATUS OnDispatchProc(PDEVICE_OBJECT pDeviceObject,

                        PIRP           pIrp)

    PEPROCESS   pEProcess;

    PETHREAD    pEThread;

    //-f--> Here we get the thread pointer.

    //      The one responsible for creating this IRP.

    pEThread = pIrp->Tail.Overlay.Thread;

    //-f--> Now we get the process that owns

    //      this thread.

    pEProcess = IoThreadToProcess(pEThread);

...

Okay, see that wonder. Now you have in your hands EPROCESS structure, that according to Microsoft documentation it is an opaque structure used internally by the operating system.

“The EPROCESS structure is an opaque data structure used internally by the operating system.”

Now what can I do with this?

Although I’m sure you’ve thought of something I could do with this structure, I have a better suggestion, and why not saying much more appropriate? Even because there might have children reading this. Despite EPROCESS mean nothing, it can still bring us some useful information. We can get the handle of the process identified by this structure and thus, get additional information from it, such as its PID. See the example below.

//-f--> ZwQueryInformationProcess from

//      Windows NT/2000 Native API Reference

//      ISBN-10: 1578701996

//      ISBN-13: 978-1578701995

NTSTATUS

NTAPI

ZwQueryInformationProcess

    IN  HANDLE            ProcessHandle,

    IN  PROCESSINFOCLASS  ProcessInformationClass,

    OUT PVOID             ProcessInformation,

    IN  ULONG             ProcessInformationLength,

    OUT PULONG            ReturnLength OPTIONAL

);

/****

***     MyGetProcessID

**

**      Get the process ID from its EPROCESS structure.

**      Please, use your imagination to create a better name

**      for this routine.

*/

NTSTATUS

MyGetProcessID(IN  PEPROCESS    pEProcess,

               OUT PHANDLE      phProcessId)

    NTSTATUS                    nts = STATUS_SUCCESS;

    HANDLE                      hProcess = NULL;

    PROCESS_BASIC_INFORMATION   ProcessInfo;

    ULONG                       ulSize;

    //-f--> Zw routines are usually called

    //      from User-Mode, so to call them

    //      from Kernel, we'll need, at least,

    //      to be at PASSIVE_LEVEL.

    ASSERT(KeGetCurrentIrql() == PASSIVE_LEVEL);

    __try

        //-f--> Initialize the output parameter.

        *phProcessId = 0;

        //-f--> Now we get the handle of the process

        //      identified by this EPROCESS pointer.

        nts = ObOpenObjectByPointer(pEProcess,

                                    OBJ_KERNEL_HANDLE,

                                    NULL,

0,

                                    NULL,

                                    KernelMode,

                                    &hProcess);

        if (!NT_SUCCESS(nts))

            ASSERT(FALSE);

            ExRaiseStatus(nts);

        //-f--> To use non-documented API, it is enough to

        //      declare its prototype as it was made in the

        //      beginning os this example

        nts = ZwQueryInformationProcess(hProcess,

                                        ProcessBasicInformation,

                                        &ProcessInfo,

                                        sizeof(ProcessInfo),

                                        &ulSize);

        if (NT_SUCCESS(nts))

            ASSERT(FALSE);

            ExRaiseStatus(nts);

        //-f--> Everybody is alive so far; now just set the output

        //      parameter with the correct information.

        *phProcessId = (HANDLE)ProcessInfo.UniqueProcessId;

    __except(EXCEPTION_EXECUTE_HANDLER)

        //-f--> Ops... Something got wrong!

        nts = GetExceptionCode();

    //-f--> Release the process handle that we got.

    //      In this way, your manager will not want to kill you

    //      when, although the processes have finished,

    //      there will still be EPROCESS structures spread in RAM.

    if (hProcess)

        ZwClose(hProcess);

    //-f--> And all lived happily ever after.

    //      (including your manager)

    return nts;

You can get extensive information from the process handle. There are even ways to get the full process Path from its handle, but let’s leave that game to the next post.

See you… 🙂

February 5, 2007

At the beginning…

Writing DriverEntry routine

Writing Dispatch Functions

Getting parameters from IRP.

When my driver is unloaded

The wonderful world of Userland

Installing and testing

Configuring a virtual machine

Configuring the TARGET machine

Configuring the HOST machine

Connecting…

But hardware that matters…

Drivers for Windows course at Sao Paulo

File System Training at OSR

Computer Engineering

But I don’t have two processors, synchronism for what?

Using ERESOURCE

But I am already a big boy and I already know how to build linked lists in C/C++. Why should I use DDK resources?

What is the point on saving only nodes?

Watching IRPs

Now what can I do with this?