In the last post I gave a brief introduction about some points regarding virtual memory that I consider to be the most relevant to driver developers. With that small load of knowledge, it will be simpler to explain the differences between the methods of transferring data between applications and drivers, as well as other issues, such as interrupt servicing, memory mapping and execution context.
If we think in a very simplified way (and I do mean simplified), drivers are basically the modules that extract data from devices and make it available to applications and vice versa. From this point of view, it really does seem that writing drivers is easy. It even looks like a way of doing a memcpy from software to hardware. Today we will see a little about the ways a driver can choose to do this data transfer.
Using a system Buffer
The first method is the so-called Buffered I/O. This method uses a system buffer to transfer data between the application and the driver. When an application calls the WriteFile function passing the data to be sent to the driver, the I/O Manager makes a copy of the application’s data into a system buffer. But what is a system buffer? It is an allocation in System Space that was made in Kernel-Mode, and for that reason, is accessible in any process context. We already talked about this in the last post.
BOOL WINAPI WriteFile(
__in HANDLE hFile,
__in LPCVOID lpBuffer,
__in DWORD nNumberOfBytesToWrite,
__out_opt LPDWORD lpNumberOfBytesWritten,
__inout_opt LPOVERLAPPED lpOverlapped
);
The I/O Manager gets the size of the buffer to be allocated from the nNumberOfBytesToWrite parameter, then performs a non-paged memory allocation and copies the data that is in User Space to System Space. If you are slipping on the terms System Space, User Space and types of memory allocation, take a read of this post so that things become less obscure for you.
In the driver, the pointer to this buffer is obtained from the IRP and its size is obtained from the IRP’s stack location. Cool, but what is an IRP? Take a look at this mini example below to get an idea, or you can see the example used in this other post that also uses the Buffered method to send and receive strings from an example driver.
//-f--> Handler routine for IRP_MJ_WRITE.
// Example of obtaining the system buffer
// allocated by the I/O Manager in the Buffered method
NTSTATUS
OnDispatchWrite(__in PDEVICE_OBJECT pDeviceObj,
__in PIRP pIrp)
{
PIO_STACK_LOCATION pStack;
PVOID pBuffer;
ULONG ulLength;
//-f--> Gets the current stack location of the IRP
pStack = IoGetCurrentIrpStackLocation();
//-f--> Here we get the pointer to the buffer
pBuffer = pIrp->AssociatedIrp.SystemBuffer;
//-f--> Here the size of the buffer
ulLength = pStack->Parameters.Write.Length;
...
}
In read operations, the method is very similar, but although in this case the I/O Manager also makes the allocation in non-paged memory, it does not make any copy into the system buffer. The system buffer is received by the driver also through pIrp->AssociatedIrp.SystemBuffer, but its size is obtained from pStack->Parameters.Read.Length. The driver will fill the buffer with the data coming from the device, and the I/O Manager will copy the buffer from System Space to User Space when the IRP is completed, filling the application’s buffer.
Whoa, whoa! As far as I know, drivers normally complete IRPs in an arbitrary context, which means “only God knows in which process context”. It is fine if the driver writes to a system buffer, which is valid for any process, but the application’s buffer is in User Space and is only accessible in the context of its own process. How would the I/O Manager copy the system buffer to the application buffer in an arbitrary context? Wow, that really was an excellent question. I do not think even I would have imagined such a good question. IRPs can be handled both synchronously and asynchronously. The I/O Manager knows how the IRP was processed, and in the synchronous case, the I/O Manager is already in the context of the process that made the request, and in this case it has access to both buffers, since the I/O Manager runs in Kernel-Mode. When the IRP is handled asynchronously, the I/O Manager queues an APC (Asynchronous Procedure Call), which is an asynchronous call executed in the context of a given thread. That thread is precisely the thread that started the operation, and which therefore will be in the right process context to access the application’s User Space.
BOOL WINAPI ReadFile(
__in HANDLE hFile,
__out LPVOID lpBuffer,
__in DWORD nNumberOfBytesToRead,
__out_opt LPDWORD lpNumberOfBytesRead,
__inout_opt LPOVERLAPPED lpOverlapped
);
In the call to the ReadFile function, the nNumberOfBytesToRead parameter indicates the size of the buffer that the application is offering to the driver. The driver receives this value as the maximum number of bytes that can be returned to the application. Assuming the application has offered 1000 bytes, the I/O Manager allocates 1000 bytes and passes the buffer to the driver. Let us suppose the driver has only 500 bytes to return to the application; in this case, the I/O Manager will have to copy to the application buffer only 500 of the 1000 allocated bytes. The I/O Manager receives this value through the pIrp->IoStatus.Information field, which is filled in by the driver before the IRP is completed. This way, the I/O Manager copies only the valid bytes from the system buffer to the application buffer. This same value is returned to the application through the lpNumberOfBytesRead parameter.
Using non-paged memory to hold the system buffer assures us that the pages will not be removed from RAM by paging. This allows the page to be accessed even from threads running at a high priority level. However, the Buffered method is indicated only for small data movements. Imagine that an application wants to write 10 MB all at once. The I/O Manager would have to make a System Space allocation of 10 MB of non-paged memory, which would not be at all appropriate, since non-paged memory is a scarce resource. If the I/O Manager manages to make the allocation, it will still have to make a 10 MB copy from the application buffer to the system buffer. This would even work, but we would have serious performance problems. In this case, the most appropriate thing would be to use the method seen next.
Locking the Application’s memory
In the method called Direct I/O, as the name already suggests, the driver accesses the application’s memory pages directly without using an intermediate buffer. This way, the I/O Manager does not make an allocation in non-paged memory and also does not have to do the BPL-BPC (Buffer over there – Buffer over here). Instead, the I/O Manager tests the memory pages that make up the buffer offered by the application, creates an MDL and locks the memory pages in RAM. Whoa! Hold on there my friend, let us go slowly!
- Tests the memory pages – Nothing stops a programmer from doing something silly. An invalid buffer may be passed to the I/O Manager. It could be that the buffer was not allocated, or that the buffer is smaller than the value indicated in the call to the ReadFile or WriteFile functions; it could also be that the memory pages offered to ReadFile are protected against writing, or that some of the pages used by the buffer have been paged out to disk. If a driver tries to access an invalid or protected page, an exception is generated and a blue screen will emerge from the darkness. But how is the I/O Manager going to test the memory? If the buffer is passed as a parameter to the ReadFile function, then the driver will perform writes to this buffer. To save time, the I/O Manager will write one byte to each RAM page just to test access to them. This write is done under an exception handler. If the buffer is invalid or protected, the exception handler will handle it and return an error to the application. If the buffer is passed to the WriteFile function, then the buffer will be read by the driver, and in this case, the test would be reading one byte from each page.
- Creates an MDL – An MDL (Memory Descriptor List) is a data structure that describes the various memory pages that make up a buffer. These pages are the same physical pages used by the application, that is, when the driver writes to these pages, it will already be writing directly to the application’s buffer. So the I/O Manager will not need to do any BPL-BPC.
- Locks the pages in RAM – This step makes the memory pages that compose the application’s buffer become non-pageable. This way, they can be accessed by the driver in threads that are running at a high priority level.
After all this ritual, the driver now receives the buffer through pIrp->MdlAddress, but what we will have here is a pointer to an MDL. But what do I do with an MDL? Most of the time, you will pass it as an argument to a service offered by another driver or system component. Some examples are DMA (Direct Memory Access) drivers that use an MDL in the call to the MapTransfer function, or even when an MDL is passed on to USB (Universal Serial Bus) controller routines, such as UsbBuildInterruptOrBulkTransferRequest. MDLs are opaque structures, but if you want to have access to the application’s buffer, then we must call the MmGetSystemAddressForMdlSafe function to get the address of the buffer to be written/read. Note that the address returned by this function is in System Space, and thus, accessible in any process context. But is not the application buffer in User Space? Yes, but what we have here is a physical memory page being mapped both to the application’s address space and to the system address space.
The size of the buffer described by the MDL is also obtained from the IRP’s stack location, just as in the Buffered method.
Neither Buffered I/O nor Direct I/O
The third method is simply not using the first two. In the method called Neither, the I/O Manager does not make a copy into a system buffer nor build an MDL to describe the application’s buffer. When the IRP reaches your driver, you have access to the virtual address of the buffer offered by the application through pIrp->UserBuffer. This address points to User Space, and for that reason, remember that this address is only valid in the context of the process that requested the I/O. The use of this method requires more care, because your driver needs to be the first driver in the device stack, and thus, ensure that the IRP reaches your driver in the context of the process that made the I/O request.
You can check whether you are in the context of the process that requested the operation by doing the test below.
/****
*** EstouNoContextoDoProcessoQueGerouEssaIrp
**
** Function with a ridiculous name that checks whether
** we are in the context of the process that generated
** the IRP passed as a parameter.
*/
BOOLEAN EstouNoContextoDoProcessoQueGerouEssaIrp(__in PIRP pIrp)
{
PETHREAD pEThread;
PEPROCESS pEProcess;
//-f--> Gets the thread that generated the IRP
pEThread = pIrp->Tail.Overlay.Thread;
//-f--> Gets the process the thread belongs to
pEProcess = IoThreadToProcess(pEThread);
//-f--> Here we compare the current process with
// the process that generated the IRP
return (PsGetCurrentProcess() == pEProcess);
}
The fact of being in the correct process context does not guarantee that the buffer passed as a parameter is valid. Unlike the Direct method, at this point the I/O Manager did not test the buffer before passing the IRP to the driver. We will have to do this ourselves. Remember that, just as in User-Mode, when accessing an invalid buffer the driver will receive an exception that must be handled, otherwise, all blue.
To perform the test, we will have to access one byte of each page of the buffer that was passed to us and check whether the world ends. The ProbeForRead and ProbeForWrite routines do this for us, but they must be called inside an exception handler. It is worth remembering that in a read operation, the application sends us a buffer where the driver will write data for the application. Since the driver will perform a write to this buffer, we will have to perform a write test (ProbeForWrite) in read operations (ReadFile). Analogously, the driver must perform a read test (ProbeForRead) in write operations (WriteFile). Take a look at the example read routine that follows below. As you should already be used to, read the comments that complement the text.
/****
*** OnDispatchRead
**
** Another function with a silly little example name.
** Validates the buffer sent by the application through
** the Neither method and writes a numeric sequence
** into the application's buffer.
*/
NTSTATUS OnDispatchRead(__in PDEVICE_OBJECT pDeviceObj,
__in PIRP pIrp)
{
PIO_STACK_LOCATION pStack;
PVOID pBuffer;
ULONG ulLength;
//-f--> Checks whether we are in the context of the process
// that generated this IRP
if (!EstouNoContextoDoProcessoQueGerouEssaIrp(pIrp))
{
//-f--> Signals to the I/O Manager that the house fell down
pIrp->IoStatus.Status = STATUS_INVALID_PARAMETER;
pIrp->IoStatus.Information = 0;
//-f--> Completes the IRP with failure
IoCompleteRequest(pIrp, IO_NO_INCREMENT);
return STATUS_INVALID_PARAMETER;
}
//-f--> Gets the current stack location
pStack = IoGetCurrentIrpStackLocation(pIrp);
//-f--> Here we get the address of the buffer offered
// by the application as well as its size.
// Note that this buffer must be in User Space
pBuffer = pIrp->UserBuffer;
ulLength = pStack->Parameters.Read.Length;
//-f--> The functions that test the buffer throw exceptions
// if the buffer is invalid. That is why we have
// to do the test inside an exception handler
__try
{
//-f--> If you take a look at the reference, you will see that
// this routine returns VOID, so the only
// way to know whether the buffer is invalid is
// by handling the exception that will be generated.
ProbeForWrite(pBuffer,
ulLength,
1);
}
__except(EXCEPTION_EXECUTE_HANDLER)
{
NTSTATUS nts;
//-f--> Oops! Invalid buffer.
// Let us get the exception code and put an
// end to this suffering
nts = GetExceptionCode();
pIrp->IoStatus.Status = nts;
pIrp->IoStatus.Information = 0;
IoCompleteRequest(pIrp, IO_NO_INCREMENT);
return nts;
}
//-f--> Here we know the buffer is safe to receive
// writes. Let us just write a numeric sequence
// just to...
for (ULONG i = 0; i < ulLength, i++)
((PUCHAR)pBuffer)[i] = (UCHAR)i;
//-f--> Here we report that the operation was carried out
// successfully.
pIrp->IoStatus.Status = STATUS_SUCCESS;
//-f--> Even though in the Neither method the I/O Manager does not
// use this number to do BPL-BPC, this
// number is still returned by the ReadFile function
// to tell the application how many bytes of the buffer
// are valid for the application to read.
pIrp->IoStatus.Information = ulLength;
//-f--> Gives the IRP a Fatality
IoCompleteRequest(pIrp, IO_NO_INCREMENT);
return STATUS_SUCCESS;
}
For all the methods, it is important to note that setting the pIrp->IoStatus.Information field tells the application how many bytes were read or written in the buffer, regardless of whether or not there is a system buffer copy as in the case of the Buffered method.
Another thing that will not change for the different methods is obtaining the size of the buffer offered by the application. This value always comes through the stack location, as already seen in the methods already discussed.
How to select the method
It makes total sense for the choice of method to be made before the first IRP reaches the driver. This is done right after the device is created. After the call to the IoCreateDevice function finishes, we receive the new device through an output pointer. The Flags member of the DEVICE_OBJECT structure is a bit mask and the DO_BUFFERED_IO and DO_DIRECT_IO bits configure the transfer method.
So it is that easy. To configure the Buffered method, we set the DO_BUFFERED_IO bit; to configure the Direct method, we set the DO_DIRECT_IO bit; and finally to set the Neither method, we do not set either of these bits. I have read in some book, which I cannot find now, that the behavior is undefined if you set both bits.
Here is another silly little example of how to set the device that has just been created for transfers in the Buffered method.
//-f--> Creates the device that will receive the IRPs
nts = IoCreateDevice(pDriverObj,
0,
&usDeviceName,
FILE_DEVICE_UNKNOWN,
0,
FALSE,
&pDeviceObj);
//-f--> Checks whether the device was created
if (!NT_SUCCESS(nts))
return nts;
//-f--> Here we can already configure the desired
// transfer method, which in this
// example is Buffered I/O
pDeviceObj->Flags |= DO_BUFFERED_IO;
But what if an IRP is delivered to the driver before we set these bits? Another excellent question. Things happen like this. Whenever a new device is created, the DO_DEVICE_INITIALIZING bit is set. IRPs only start being delivered to this device when the driver clears this bit. This allows us to initialize the device before any IRP arrives.
Nice try, smart guy, but your example does not clear this bit. How do you explain that? You are impossible today! At the end of the DriverEntry routine, when control returns to the I/O Manager, it scans the list of devices created by the driver and clears this bit for us. It is important to remember that we still need to clear this bit when we create a new device after the DriverEntry routine has finished. A very common example is WDM devices, which are created in the AddDevice routine, but that will be for another time. This post has already gotten very long.
See you! 🙂
Leave a Reply