Step into Kernel (SoftIce)

December 23rd, 2006 - Fernando Roberto

Finally back from vacation. It has been nearly six years that I did not take vacations. Traveling, visiting new places and just relaxing were my main tasks during these last 20 days. Back to work, I’ve just arrived at the company and I was amazed by it. I could not believe they sent that old one away. We exchanged glances and obviously I got butterflies in my stomach. I got close enough to be able to smell it. I looked up and down. Simply perfect and it was an injustice to leave it alone. It didn’t take long and it was under my control. I had to prove everything it had to offer. After all, there were so many choices: Coffee, cappuccino, chocolate, tea with or without sugar. But it was all an illusion. “Oh my… that’s was a horrible coffee.” My friends, appearances can be deceiving. How can so beautiful machine make so bad coffee? I confess I miss the old machine. I didn’t mean that coffee was not so good, because the coffee was really horrible! Now I’m looking for alternative ways to keep my addiction. Perhaps there are caffeine patches to sell out there.

Don’t worry, this will no longer be an “Off Topic” post about changing coffee machines and the impact caused by them. In this post, which I had started last year and have just finished it now, I’ll take the steps needed to use SoftICE system debugger.

I still was new in SCUA and I was developing a GINA for our security product. For those who do not know, Gina is one small screen that appears during the Logon in NT platform. GINA is a DLL that is loaded by winlogon.exe process, which has the function of making the user’s interface to perform the login in the computer. The Msgina.dll GINA is the original one, but we can build our own GINA to provide alternative ways of authentication. A practical example of this is the product True Finger sold by SCUA. This product changes the system’s GINA, so that, with a biometric sensor help, it makes the system ask for the user’s fingerprint to logon the system. Debug GINA requires a few atypical steps, since to test it is necessary to do Logon, Logoff and things like that. You can also debug it using the Visual Studio Remote Debug, but at that time, debugging GINA was the need I had to learn how to use SoftICE.

There were two development teams, one for Kernel drivers and the other for applications. I still worked on application staff when in a meeting area, the team introduced me to SoftICE as an alternative to solve the problems about Kernel Debug I was facing. I was using techniques such as log files and MessageBox to debug GINA. You can imagine the MessageBox with the typical messages as “I called the X routine”, “Returning from Y routine.” After I learned how to use SoftICE, it solved many things for me. Specially since other Ginas had to be developed and as, the time passed by, I started to develop kernel drivers.

Most of the application team had a panicked about using SoftICE. A situation very similar to that described in my previous post. They said: “Oh no, using SoftICE is too complicated. I will continue with the MessageBox. If I cannot solve this, then I will use a Kernel Debugger.” I think it was just about getting used to it, SoftICE is not a rocket science and nor difficult to use. I found it amusing when one part of the team was caught trying to solve a particular problem, that kind of strange bug that nobody could find. I went around and asked: “Why don’t you just use SoftICE?”, “No, no, I guess it is not necessary.” they said. Later I also was called to help tracing the problem, and in most of the time I could, I’d picked up the problem that they’d chased for days in half an hour of Debug. I’m not saying I’m super Debug genius, or that they were technically incompetent. I’m saying that it is only a matter of using the right tool. You could dig a tunnel with a teaspoon, but with a shovel things would be much easier.

SoftICE is part of a development package called Driver Studio, which was sold by Compuware. For those who still do not know, Compuware has discontinued SoftICE and we cannot count on a version of this tool for Windows Vista. With SoftICE is possible to debug the kernel using only one machine, but the product still has the resources to do remote debug via serial ports or network adapters. Thus, they are also adopted the names Host for the machine that does the Debug and Target for the machine that will suffer the Debug (Similar to WinDbg). Here we will adopt the Host name for identifying the machine where we have the compiler and the installation type Host SoftICE. We will not do any remote debug on this post.

Very briefly, we can say that SoftICE works as it follows. At the Host machine, it is necessary to create a symbol file known as NMS. This file is copied along with the driver to be tested to the Target machine. In it, we load the symbols and start debugging the driver. We will describe these steps in more details next.

Installing the Host Machine

Installing SoftICE on the development machine is necessary for us to generate the NMS file. Don’t worry; you will not test your driver on your development machine. At least not while you’re sane. During the setup process, you will pass through the typical setup screens until you reach the screen shown above where you select installation type you want. Because we use only installation Host portion, select that option and skip to the next screen. Select only the option “SoftICE Host Components”, so that it appears on screen as below. The remainder of this process is next, next…

Installing the Target Machine

We will use the test machine in a way that it is not necessary to have two machines to do the debug, and so we will explore this feature that is usually used. To do this, select the “Full Instalation” in the first screen, then select only “SoftICE Host Components” items and “SoftICE Target Components” as it is shown below.

In this case, at the end of the register and file copy process, we will have to set some debugger properties. The first group of settings refers to initialization. We will select the Manual startup at this moment. All these settings can be changed after the product is installed. The setup screen should be as shown below.

The next group will be about general settings. In this item, we will change the field “Initialization” to stay with the configuration shown below. These are commands that are executed when the debugger is started. Each command is separated by a semicolon. These commands will make the following changes:

  • LINES 60; Change the total amount of lines to 60
  • WC 30; Change code window size to 30 lines
  • WL 10; Change local variable window size to 10 lines
  • X; And finally, the command that starts the system normal execution

The lack of “X” command will cause the system to become paralyzed at debugger startup time. This gives us the opportunity to put initial breakpoints and later we will be able to continue with the system execution by pressing F5, which is the hot-key for “X” command.

Now let’s set the video configuration. Right now you can ask me: “What do you mean by video configuration? Doesn’t Windows implement an abstraction layer called GDI?”

SoftICE is a system Kernel debugger. GDI is part of the system. How would you debug a video driver if the debugger itself utilizes this driver? The debugger should influence the debugged system as little as possible. It is for this reason that WinDbg need a serial cable and another whole machine to make the Kernel Debug possible. Similarly, SoftICE, once loaded, cannot use any system resource. This includes video drivers, disk, keyboard, mouse and so on. For the SoftICE being able to have a place in the video while debugging the system, it accesses the video memory directly and draws manually its entire interface. That’s why we have to configure the video.

This configuration can be chosen between two main options. The first is the “Universal Video Driver”, where the interface will appear in a window. This “window” is actually the result of writing in video memory that gives the impression of a window. This is not the type of window you’re used to seeing and drag with the mouse or even suffer the action of ALT + TAB. Remember, when SoftICE displays its user interface, the whole system is frozen. The other option is “Full screen”, where SoftICE switches the video to text mode to display its interface.

If you are installing SoftICE on a virtual machine, some steps are required:

  • Do not install tools such as video acceleration like VmTools.
  • Use the standard VGA driver in Windows and configure it so that, it is 640×480 with 16 colors. (Disgusting but necessary)
  • Select the “Full Screen” option in SoftICE.

If these steps are not followed, the machine will freeze when asked, but the debugger screen will not appear. Assuming that we are installing on a real machine, select the video option as shown below.

The remaining is next, next…

Generating the NMS File

On the Host machine, after compiling your driver (if you still don’t know how to do it, read this other post), you must now run the application “SoftICE Symbol Loader” that was already installed. This application does the symbol translation and generates the NMS file. Select Open… from the File menu and point out the driver you will debug. Once opened, the program must place the driver loaded name at the window title. I have numbered the three buttons you need for this operation.

  • Make sure that the second button is pressed. This will make any driver source code be appended to the resulting NMS.
  • Also make sure that the “3” button is released. This will prevent the program for asking for sources that it cannot find.
  • Click on “1” button that will do the symbol translation and the program will generate NMS file at the local directory.

Starting SoftICE

After installing your driver as usual in the Target machine, we will be ready to start the Debug of our driver. Remember that both our sample driver and SoftICE are configured to start manually. To start SoftICE, you can click the “Start SoftICE” icon that was installed in your Start menu or simply run the command “net start ntice” in the “Run…” window. To make sure that the debugger has loaded, press CTRL + D so that the system freezes and the screen is shown as the one below.

No, this is not a CRASH DUMP of your video card that has just exploded. Gentlemen, I am pleased to introduce you the SoftICE. Now can you understand the application staff panic? Don’t worry, it’s ugly but it doesn’t bite you. Pressing F5, the system returns to normal execution.

Now we have to load the NMS file into the debugger. This is easily done simply by double-clicking the NMA file. This file can be in any directory on the machine. The only time that this file is used is when you load it in the debugger. To verify whether the file was successfully loaded by SoftICE, use the “table” command that lists all loaded symbol tables. Several tables can be loaded simultaneously, but only one is active at a time. The “file *” command lists all files of a given table. To open a specific file, enter the file name in front of the command. See example below.

With the file already opened in the code window, press ALT + C to put the cursor into the source code. If this key sequence is not working, press ESC and then ALT + C. Note that you can only walk with the cursor up and down. Attempts to move the cursor sideways causes the command window to gain focus again. Place the cursor on the line you want to insert the breakpoint and press F9. At this time, the line was highlighted as shown in the picture below.

Now let’s press F5 to let the system return to normal operation. At the moment the driver starts, our breakpoint will interrupt this action and the SoftICE interface will be displayed. The highlighted line shows what the next instruction to be executed. Pressing ALT + L, the local variables window gains focus. You can navigate among the variables by moving the cursor up and down. Press ENTER key on the structures that begin with “+”, so that they are expanded. The image below shows register window and then, the local variable window.

I will leave a small table of commands here, but the full reference is in “SoftICE Command Reference” PDF file.

  • F5 : Run
  • F9 : Breakpoint
  • F8 : Step into
  • F10 : Step over
  • F12 : Step out
  • F7 : Run to cursor
  • WL : Local variable window
  • WW : Watch window
  • WD : Memory window
  • F3 : Swap among C source/Assembly/Mixed

Obviously there are numerous details and technical use, but let’s leave that to the documentations that coming with the product.

Wow, what a long post! You thought it would be easy, didn’t you? Until the next time …

Leave a Reply