Archive for October, 2006

Serial Killers

Tuesday, October 24th, 2006

There’s no doubt, they’re really giving up producing serial ports on notebooks and desktops. In this post, I will talk about some problems we encounter when debugging that driver that only reproduces the issue on that machine that has no serial ports. What alternatives do we have with this regard?

This month new computers have arrived here in the company. In these machines, we put some of our products in beta version. For any reason, only on new machines one of our drivers was not working as it should. As the quotes of the programmer’s code of ethics, the blame is on the trainee who did the tests until otherwise could be proved. After several unsuccessful attempts to discover what was happening, I had no doubts, let’s debug it. Avoiding moving the victim to my desk to connect the serial cable, I decided to install SoftIce. But life is a surprise box and we got a nice BSOD when SoftIce was started. Wow! This is rare, but it happens. In these cases, we don’t have much choice; we really have to use Windbg. The PCs we got were Dell Dimension 5150 model. Take a look at its cabinet back in the picture and answer quickly: where is the serial port? That’s it, there is none. By consulting the manufacturer’s Web site, we found that the computer had 8 USB ports but had neither a parallel nor a serial port.

At this moment, we made a circle and talk all together: “Oh God, what do we do now?” Some issues that happen in real machines are also reproduced in virtual ones, especially if the driver you are developing is a filter or a driver that does not deal directly with the hardware. Just for a test, we installed VMware on a new machine and fortunately the problem was reproduced. That’s one of those occasions that we get happy when the problem appears. Thereafter, I just used the old trick about having virtual serial port in a VM and redirect the data stream to a named pipe. This could become a post. 😉

Well, after connecting the debugger, having the break-point set, it did not take long to find the problem, generating a new version, testing, making it to work and live happily ever after. But not always the history is solved with these few steps. Some virtual machines do not support USB devices. Even VMware that offers this feature still does not support human interface devices (HID) like USB keyboards and mice. Not to mention the psychological and esoteric issues involving race conditions and/or a good joint action with Murphy. What can we do in these cases besides sitting and crying?

Fortunately, technology at the service of humanity predicted situations like these. In these cases, we traditionally have two alternatives:

1) Describe the bug you want to delete on a sheet of paper. It is recommended that this description has sections like “Go away strange bug …”. Sew a piece of paper inside a frog’s mouth and positive thinking in solving the problem; throw the frog into a river without seeing where it would fell. After seven days, do a “Rebuild All” in your project (A reboot before the build is recommended). If the problem persists, repeat the steps. Formatting the machine helps to remove bad fluids and scare evil and haunt spirits away that torment your code. Consult a software healer for best results.

2) Use a FireWire card and make a Windbg connection using an IEEE 1394 cable. This option is only available if the machine being debugged is a Windows XP or higher. These cards are installed on your PCI or PCMCIA bus. I did not have any chance of using this connection type to debug a machine, but it is certainly very fast and comfortable, since the data is transmitted at a rate of up to 400Mbps.

Can I use one of those famous serial ports to USB converters that are sold at Santa Ifigenia? Although it’s not required to install a driver on your system, these adapters use drivers to work. All USB stack is mounted to make these devices work. On a machine that is being debugged in Kernel, all the code that deals with serial communication or firewire is hard-coded in the system and it works with fixed standard addresses . These adapters can be used without problems by the debugger machine, but not by the debugee one.

What if I use a PCI multiport serial card? That depends on what I/O addresses and interrupts the card. Normally these cards use different addresses of the standards in order to not to create any compatibility issues with I/O addresses that probably pre-exist on the machine. If the card offers the option of using standard addresses, there is no problem.

Windows Vista will support Kernel Debug via USB 2.0 port, though some conditions are required. USB cable must be special for this purpose, the debugee computer USB interface must support Kernel Debug, and last but not least, the debugger computer USB controller should be Extended Host Controller Interface (EHCI) compliant.

If you are interested in acquiring IEEE 1394, remember that you must have, at least, two interfaces such that there is a link between the debugger and debugee machine. Although I have seen some lost comments on the Internet about FireWire x USB converter cables (for use in debugger machine), as far as I could see, they are electrically impossible to be built. That’s because the entire protocol is different. Whereas USB is a “Master x Slave” protocol, FireWire is “Peer to Peer” one. It would require some intelligent hardware along the way to do all translation.

The company I work for is acquiring these interfaces for PCMCIA FireWire in order to eliminate the problem of debugging customer’s notebook  that is crippled serial ports. I think soon we’ll give up the frog and all. I will make some comparisons with the good old serial cable and give some news on this subject.

Driver plus plus

Tuesday, October 10th, 2006

Developing low-level software is really good. I personally love the kind of work I do. I have close friends who marvel at C++ beauty, how things appear to fit perfectly among STL, iterators and templates. My business is definitely another. Despite using C/C++, Assembly and appreciating these languages a lot, I don’t deeply dedicate myself to study C++. My practical side to solve problems leads me to do some things that would make my fried Slug want to beat me, but fortunately he is a calm person. 😛

After my brother (Kabloc) had read my post Getting Started…, he asked me whether it’s be possible tp use C++ instead of C for developing drivers. In this post I will summarize the problems to get an object-oriented Blue Screen. I will not opine whether C++ is better or worse than C to write drivers. This is a discussion that seems to have no end. If you participate in any discussion list about driver development, you know what I mean. I personally prefer create classes, namespaces and templates that address in a modular way to my problems. That seems to be more comfortable. In fact, we can feel a certain tendency in Kernel Mode development for object-oriented programming. This is something increasingly present with DDIs (device-driver interfaces) using COM model for communication between layers in Kernel and the coming of WDF. Remember that using COM model does not mean using COM, but only the concept of derived interfaces well known as IUnknown to implement reference counters and exchange interfaces.

First of all, let us build the sample project created in my previous post to show a driver that makes use of a class. As the tradition dictates, we’ll start thinking it’s easy and simply use the CPP file extension at source below and see what happens.

//-f--> A class with a very original name
class MyClass
        DbgPrint("Constructor %p\n", this);
        DbgPrint("Destructor %p\n", this);
    VOID SayHello(VOID)
        DbgPrint("Hello from %p\n", this);
**      DriverEntry 
NTSTATUS DriverEntry(IN PDRIVER_OBJECT  pDriverObject,
                     IN PUNICODE_STRING pusRegistryPath)
    MyClass Instance;
    //-f--> A life sign
    DbgPrint("DriverEntry called\n");
    //-f--> Hello...
    //-f--> Everything is fine so far
    return STATUS_SUCCESS;

Round #1…

Compiling - krnclass.cpp for i386
...\ntddk.h(2152) : error C2220: warning treated as error - no object
                    file generated
...\ntddk.h(2152) : error C4162: '_ReturnAddress' : no function with
                    C linkage found
...\ntddk.h(6889) : error C4162: '_InterlockedExchange' : no function
                    with C linkage found
...\ntddk.h(6915) : error C4162: '_InterlockedIncrement' : no function
                    with C linkage found
...\ntddk.h(6928) : error C4162: '_InterlockedDecrement' : no function
                    with C linkage found
...\ntddk.h(6942) : error C4162: '_InterlockedExchangeAdd' : no function
                    with C linkage found
...\ntddk.h(6972) : error C4162: '_InterlockedCompareExchange' : no
                    function with C linkage found
...\ntddk.h(7024) : error C4162: '_InterlockedOr' : no function with C
                    linkage found
...\ntddk.h(7034) : error C4162: '_InterlockedAnd' : no function with C
                    linkage found
...\ntddk.h(7044) : error C4162: '_InterlockedXor' : no function with C
                    linkage found

We find some definitions in ntddk.h that need to be compiled in C, and not in C++. This problem will also happen with the driver’s entry point DriverEntry. DDK expects to find this exported symbol as C. But this is still very easy to solve, just change the way we include this header and the way we declare our entry point so, that it’ll look as shown below.

extern "C"
extern "C"
NTSTATUS DriverEntry(IN PDRIVER_OBJECT  pDriverObject,
                     IN PUNICODE_STRING pusRegistryPath)

So, it has compiled properly. Loading this driver, we can see the following messages in DebugMon.

Are we done? Do we already have a driver in C++? Not yet. We can already enjoy some concepts like classes, templates and overloads, for example, but let’s see what happens when we try to use the new operator.

extern "C"
NTSTATUS DriverEntry(IN PDRIVER_OBJECT  pDriverObject,
                     IN PUNICODE_STRING pusRegistryPath)
    MyClass *pInstance = NULL;
    //-f--> Creating an instance dynamically
    pInstance = new MyClass();
    //-f--> Crashphobia
    if (!pInstance)
        return STATUS_NO_MEMORY;
    //-f--> Hello from heap
    //-f--> Releasing
    delete pInstance;
    //-f--> Everything is fine so far
    return STATUS_SUCCESS;

Round #2…

Linking Executable - objchk_wxp_x86\i386\krnclass.sys for i386
krnclass.obj : error LNK2019: unresolved external symbol "void * __cdecl
               operator new(unsigned int)" (??2@YAPAXI@Z) referenced in
               function _DriverEntry@8
krnclass.obj : error LNK2019: unresolved external symbol "void __cdecl
               operator delete(void *)" (??3@YAXPAX@Z) referenced in
               function "public: void * __thiscall MyClass::`scalar
               deleting destructor'(unsigned int)" (??_GMyClass@@QAEPAXI@Z)
objchk_wxp_x86\i386\krnclass.sys : error LNK1120: 2 unresolved externals

Oops! To resolve these dependencies, we have to overload new and delete operators so, that allocations would be made through the ExAllocatePool(). Let’s pass an additional parameter to the new operator so that we can choose between NonPagedPool, PagedPool and so on …

typedef unsigned int size_t;
//-f--> Overloading the new operator
__inline void* __cdecl operator new(size_t size,
                                    POOL_TYPE pool)
    return ExAllocatePool(pool, size);
//-f--> Overloading the delete operator
__inline void __cdecl operator delete(void *pVoid)
    //-f--> We should specify the pool type here
    pInstance = new(NonPagedPool) MyClass();

Now it compiles again and the memory allocations are made in the way it should do, but there is another problem that happens when you try to create static or global objects. Let’s demonstrate this by creating a global instance and again, trying to compile this example.

Round #3…

Linking Executable - objchk_wxp_x86\i386\krnclass.sys for i386
krnclass.obj : error LNK2019: unresolved external symbol _atexit
               referenced in function _$E1
objchk_wxp_x86\i386\krnclass.sys : error LNK1120: 1 unresolved externals

Since we have no support to C++ in Kernel, we have no run-time that implements all static object initialization by calling all global constructors and destructors of them. This is a process somewhat complicated and it will require some more concepts. There is a very good article from Matt Pietrek that describes this process. Looks like this will give us a lot of work, but fortunately there is a MSI that brings an implementation of Kernel C++ run-time at Hollis Technology Solutions site . The package is an open source, and thus, it brings all the source code with it (including an example) and the libraries of this implementation.

This run-time implementation is broader than described in this post and we will have enough to play with C++ in Kernel. For C users, it brings even the definitions of malloc() and free(). The only uncomfortable thing I found in this package is a need to replace the definition of our DriverEntry() entry point to CPP_DRIVER_ENTRY.

**      DriverEntry 
                 IN PUNICODE_STRING pusRegistryPath)

I’ll leave an example for downloading about how to use this package, admitting that you have installed MSI on C:\Library\HtsCpp. This example may also serve as the basis for future posts that would use object orientation feature. Anyway, the following is a preview of how your SOURCES file should be.

!if "$(DDKBUILDENV)" == "fre"

See you! 🙂

RtlGetModuleBase & RtlGetProcAddress

Monday, October 2nd, 2006

In the post ExAllocatePool (WithoutTag), I talked a little about the conflicts on using new APIs in drivers and making them incompatible with legacy systems. Our initial proposal was to have a single binary that would be able to run either in Windows NT and newer systems. We reached a solution that cannot be considered ideal, that old functions are always used on systems that support newer APIs.

Thinking about solving this limitation, the DDK brought us MmGetSystemRoutineAddress(). Analogous to GetProcAddress() exported by kernel32.dll, MmGetSystemRoutineAddress() dynamically obtains a function address from its exported name. But the world will not still be safe while Windows NT survives. This new function has only been implemented since Windows 2000.

We don’t have an alternative to Windows NT, so let’s make one. With one or two PE concepts, we implemented this function version. PE structure carries tons of rules, but for our need we can implement a light version. If you want details about the rules adopted by PE, you can take a look at the article An In-Depth Look into the Win32 Portable Executable File Format by Matt Pietrek.

Well, now that we already have read the entire article and already known everything about PE, follow the basic algorithm prototype about how to walk through PE structure, looking for a specific function exported by name. This function is not provided by Windows NT DDK and is written in the example source code available for downloading.

NTSTATUS RtlGetProcAddress(IN PVOID     pBaseAddress,
                           IN LPCSTR    pszFunctionName,
                           IN NTPROC    *pProcedure);

Notice that the input parameters are the base address, the name of desired function and ultimately the address to the pointer that will get the obtained function address. But where will I get this base address from? Remember the functions we are looking for are exported by ntoskrnl.lib and they are implemented in the ntoskrnl.exe module. To make sure the function we’re looking for is in fact exported by this module, use the “Dependency Walker” to see this module export table. As I said before, some functions are implemented as macros, thus, its definition will be in a header file and not in the export table.

To get the module base address that exports these functions, we use ZwQuerySystemInformation() in which, although not documented by Microsoft, there are already several publications that comment about it. Thus, we’ll define the RtlGetModuleBase() which will behave similar to the well-known GetModuleHandle(). This function is also defined in the example available for downloading.

ZwQuerySystemInformation(IN SYSTEM_INFORMATION_CLASS SystemInformationClass,
                         IN OUT PVOID SystemInformation,
                         IN ULONG SystemInformationLength,
                         OUT PULONG ReturnLength OPTIONAL)

There are many undocumented features and settings that can be extremely useful. The book Windows NT/2000 Native API Reference by Gary Nebbett is excellent to make use of these functions. He brings prototypes, enums, structures used in calls, descriptions of what each function does and each parameter. Here in our example, we only use the statements below for information about the modules loaded into the system address space.

typedef struct _SYSTEM_MODULE_INFORMATION   // Information Class 11
    ULONG Reserved[2];
    PVOID Base;
    ULONG Size;
    ULONG Flags;
    USHORT Index;
    USHORT Unknown;
    USHORT LoadCount;
    USHORT ModuleNameOffset;
    CHAR ImageName[256];

The structure “Base” member above brings us the module base address containing PE structure. In the sample code,  is RtlGetModuleBase() routine definition that has the following prototype.

NTSTATUS RtlGetModuleBase(IN LPCSTR     pszModuleName,
                          OUT PVOID*    ppBaseAddress);

With the union of its powers, now we can know whether a certain API is implemented in the current system and if so, get its address. So, it is perfectly possible to have a single binary that can run on both Windows NT using ExFreePool() and in later systems using ExFreePoolWithTag(). Below it is a very basic example as always. Of course, we can create a single allocation routine that would do the dirty work.

#include "GetProcAddr.h"
//-f--> Type for the ExFreePoolWithTag routine
    IN PVOID  P,
    IN ULONG  Tag 
VOID OnDriverUnload(PDRIVER_OBJECT     pDriverObj)
    //-f--> That routine, even empty, is here just
    //      to allow the driver to be unloaded.
**           There once was a Driver...
NTSTATUS DriverEntry(PDRIVER_OBJECT     pDriverObj,
                     PUNICODE_STRING    pusRegistryPath)
    NTSTATUS                    nts;
    PVOID                       pBaseAddress, pTemp;
    PF_EX_FREE_POOL_WITH_TAG    pfExFreePoolWithTag;
    //-f--> Set the unload callback routine
    pDriverObj->DriverUnload = OnDriverUnload;
    //-f--> Get the module base address
    nts = RtlGetModuleBase("ntoskrnl.exe",
    //-f--> Testing the routine's return doesn't hurt,
    //      but not doing this can kill your system.
    if (!NT_SUCCESS(nts))
        return nts;
    //-f--> Allocating memory for a test using the Free routine
    pTemp = ExAllocatePoolWithTag(NonPagedPool,
    //-f--> Get the function API address
    nts = RtlGetProcAddress(pBaseAddress,
    if (NT_SUCCESS(nts))
        //-f--> If that routine is implemented by the system, so
        //      we get a success return code here.
        //-f--> Who has no dog, hunts like a cat.
    return STATUS_SUCCESS;

Have fun! 🙂