{"id":363,"date":"2008-09-11T15:42:46","date_gmt":"2008-09-11T15:42:46","guid":{"rendered":"https:\/\/driverentry.com.br\/en\/2008\/09\/11\/a-pinch-of-virtual-memory\/"},"modified":"2026-07-29T18:03:59","modified_gmt":"2026-07-29T18:03:59","slug":"a-pinch-of-virtual-memory","status":"publish","type":"post","link":"https:\/\/driverentry.com.br\/en\/2008\/09\/11\/a-pinch-of-virtual-memory\/","title":{"rendered":"A Pinch of Virtual Memory"},"content":{"rendered":"
\"\"<\/a><\/div>\n

\"\"<\/a><\/p>\n

Now let us cut the small talk and get straight to what matters. The example I described in another post<\/a> shows how to implement a very simple driver that stores a list of strings that are sent to the driver through write operations (WriteFile<\/a>), and the same strings are returned in subsequent read operations (ReadFile<\/a>). This is a good example of how application data goes to the drivers and vice versa. There are three ways to access the data offered by applications from within a driver. In this post we are going to take a little look at virtual memory so that we can have a basis for future posts, where I will be able to explain the differences between those three ways. We are going to stick to just a few basic characteristics of virtual memory so that the things I will explain next make more sense to you, but if you want more details about Virtual Memory, you can read Chapter 7, “Memory Management”<\/em>, of Windows Internals<\/a>, or take a little look at a presentation<\/a> made by my friend Strauss<\/a>.<\/p>\n

What is a memory page<\/h3>\n

A memory page is a unit used by the hardware in memory-access protections. More about memory page protections here<\/a>. This is a hardware-defined unit that helps to subdivide memory into smaller, manageable spaces. Although there are two page sizes (a small one of 4KB, and a large one of 4MB), every reference to the page size seen in the documentation refers only to the small pages. The memory page size varies with the hardware platform. On x86<\/a> and x64<\/a> systems the pages are 4KB, whereas on IA64<\/a> systems the page is 8KB. The page size can be obtained through the PAGE_SIZE<\/strong> constant.<\/p>\n

Address space<\/h3>\n

In a very brief way, virtual memory is a mechanism that, in a joint effort between hardware and software, allows processes running on Windows to have their own address space. Address who?<\/strong> The address space, or Address Space<\/em> as we usually see it in the reference, allows each process to have a private view of the memory it is using. That is, a process cannot read or write to another process’s memory pages. This prevents a badly written program from erroneously writing to another process’s memory pages, or even to operating-system pages, thus compromising the stability of the whole system. Therefore, we can assume that a process’s address space is only accessible by the process to which it belongs.<\/p>\n

Note: It is possible to share memory between processes, but even in this case, the same physical memory page is referenced by two or more different address spaces.<\/p><\/blockquote>\n

Virtual Memory and Physical Memory<\/h3>\n

Right after the discovery of fire, memory was addressed directly in real mode<\/a>. This means that a pointer used by an application accessed the data exactly where it was physically located on the RAM memory chips. Virtual memory pages associated with processes reflect physical memory pages through virtual addressing. The bits that make up the virtual address map the physical memory you want to access. When dereferencing a pointer that contains a virtual address, the processor, transparently to the process, consults structures filled in by the operating system in order to translate the virtual address into a real one, and this way, finds the physical page where the data actually is.<\/p>\n

\"\"<\/div>\n

 <\/p>\n

The arrangement of virtual memory pages has no relation to their physical arrangement. This means that when you make a large memory allocation, which is composed of several memory pages, you get a pointer that, from the application’s point of view, points to memory pages that are linearly distributed (one after the other), whereas the physical memory pages may be scattered all over the RAM chips.<\/p>\n

Memory might not be in RAM<\/h3>\n

The memory used by applications is not limited to the amount of memory offered by the RAM chips installed on your motherboard. As processes, increasingly hungry for memory, are executed, the available space in RAM gets shared among the various processes. Memory pages accessed less frequently are removed from the RAM chips and go to disk (to the Pagefile.sys<\/em><\/strong> to be more precise), making room for another memory page that is needed at that moment. This process is called paging. When the application finally accesses the data that is on that page now on disk, the system allocates space in physical RAM to bring the page back from disk to RAM. This may result in other pages that were in RAM being paged out to disk.<\/p>\n

\"\"<\/div>\n

 <\/p>\n

For applications, the paging process is completely transparent. The developer is not the least bit worried about whether the region of memory the program is going to access is on disk or in RAM. You just access the data and the processor, together with the memory manager, sorts this out for you. Unfortunately, the story is not so rosy for driver developers. Applications always run at a low thread priority (PASSIVE_LEVEL<\/strong>), and when accessing data that is on a page on disk, the thread is interrupted by the memory manager so that it has the opportunity to trigger the File System<\/em> drivers and consequently the disk drivers, wait for the I\/O to be performed (even though it takes a gazillionth of a second, we still have to wait), and then the thread is released. Although there are people who do not believe in these things<\/a>, the NT platform is completely preemptive. This means that a thread can be interrupted by another one of higher priority. In Kernel Mode<\/em>, a thread can be at priorities higher than APC_LEVEL<\/strong>, which is the priority at which paging is performed. If the thread that is going to access the data is at a very high priority level (greater than or equal to DISPATCH_LEVEL<\/strong>), the processor will not be able to do the paging and a blue screen will bloom before your eyes. A typical example of high-priority execution is interrupt handling, but let us not stray from the subject.<\/p>\n

\"\"<\/p>\n

A page can be accessed not only by software, but also by hardware. Huh?<\/strong> An application can pass the pointer of a memory region to a driver, which in turn will use a DMA<\/a> channel to fill this buffer. Copying a buffer through the DMA process does not use CPU cycles to be performed. Filling a buffer is something so simple that even a chimpanzee can do it. So a group of chimpanzee engineers created a chip that does this, but that is beside the point now. This chip may be on the motherboard or on the very board controlled by the driver. The chip is programmed by the driver and a transfer is started. During the copy, neither the application, nor the operating system, nor even the processor are aware of what is happening. The system may then determine that the pages used in the transfer, which are not being accessed by any process, should be paged out to disk. If that happens, the DMA chip will keep transferring bytes to that physical RAM address and… Well, you know, right? To avoid this, there are ways to warn the memory manager that one or more pages must not be paged.<\/p>\n

During the development of a driver, you may want to allocate a buffer that will be used by an ISR (Interrupt Service Routine<\/a>). Since an ISR runs at high priority, we can make a memory allocation requesting a buffer that is not pageable, and thus ensure that the buffer obtained by this allocation is always in RAM. But take it easy: non-pageable memory is a scarce resource and must be used sparingly. Otherwise, some operations will stop being performed by the system due to lack of RAM, even if there is still plenty of pageable memory available.<\/p>\n

\"\"<\/div>\n

 <\/p>\n

User Space and System Space<\/h3>\n

Taking a 32-bit system as an example, a pointer is capable of addressing 4 GB<\/strong> of memory. Of these, 2 GB<\/strong> are reserved for addressing private pages for each process, that is, they will make up the private address space of each application. This range, which goes from 0x00000000<\/strong> to 0x7FFFFFFF<\/strong>, is called User Space<\/strong><\/em> and these are the addresses that applications access in User Mode<\/em>. As we saw, this range makes up the private address space of a process, protected against improper access by other processes. An example would be: Only the threads of Process A<\/em> will have access to the User Space<\/em> of Process A<\/em>. The same address points to different physical RAM pages in different processes, depending on the context of the process making the access. The other remaining 2 GB of the 32-bit addressing are reserved for addressing system pages. The address range from 0x80000000<\/strong> to 0xFFFFFFFF<\/strong> defines the so-called System Space<\/strong><\/em>, which, unlike User Space<\/em>, addresses memory pages that are not<\/strong> private to a given process. This means that the same data (on the same physical page) can be accessed through the same virtual address in different processes. Data in System Space<\/em> can be accessed only in Kernel Mode<\/em>, whereas data in User Space<\/em> can be accessed both in Kernel Mode<\/em> and in User Mode<\/em>.<\/p>\n

\"\"<\/div>\n

 <\/p>\n

Good grief! Was that a machine gun of concepts?<\/strong> If you show symptoms of blurred vision or vomiting followed by diarrhea and trembling, quit being a wimp and get ready for the next posts. In case you have not shown any of these symptoms, do not worry, for some people the process takes longer. In future posts I will explain how drivers access the virtual addresses offered by applications, how they deal with memory paging, and even test buffers offered by applications. A badly written driver can allow an invalid parameter in an application to cause a blue screen.<\/p>\n

That is enough for today, but it is worth remembering that what was presented here is just the tip of the iceberg<\/em>. This little tip is what I believe to be the most relevant for driver development, but there are still tons of details regarding virtual memory.<\/p>\n

Have fun! \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"

Now let us cut the small talk and get straight to what matters. The example I described in another post shows how to implement a very simple driver that stores a list of strings that are sent to the driver through write operations (WriteFile), and the same strings are returned in subsequent read operations (ReadFile). […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"pagelayer_contact_templates":[],"_pagelayer_content":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-363","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/posts\/363","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/comments?post=363"}],"version-history":[{"count":2,"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/posts\/363\/revisions"}],"predecessor-version":[{"id":365,"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/posts\/363\/revisions\/365"}],"wp:attachment":[{"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/media?parent=363"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/categories?post=363"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/driverentry.com.br\/en\/wp-json\/wp\/v2\/tags?post=363"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}