Tutorial 28: Win32 Debug API Part 1

In this tutorial, you'll learn what Win32 offers to developers regarding debugging primitives. You'll know how to debug a process when you're finished with this tutorial.
Download the example.

Theory:

Win32 has several APIs that allow programmers to use some of the powers of a debugger. They are called Win32 Debug APIs or primitives. With them, you can:

Load a program or attach to a running program for debugging
Obtain low-level information about the program you're debugging, such as process ID, address of entrypoint, image base and so on.
Be notified of debugging-related events such as when a process/thread starts/exits, DLLs are loaded/unloaded etc.
Modify the process/thread being debugged

In short, you can code a simple debugger with those APIs. Since this subject is vast, I divide it into several managable parts: this tutorial being the first part. I'll explain the basic concepts and general framework for using Win32 Debug APIs in this tutorial.
The steps in using Win32 Debug APIs are:

Create a process or attach your program to a running process. This is the first step in using Win32 Debug APIs. Since your program will act as a debugger, you need a program to debug. The program being debugged is called a debuggee. You can acquire a debuggee in two ways:
- You can create the debuggee process yourself with CreateProcess. In order to create a process for debugging, you must specify the DEBUG_PROCESS flag. This flag tells Windows that we want to debug the process. Windows will send notifications of important debugging-related events (debug events) that occur in the debuggee to your program. The debuggee process will be immediately suspended until your program is ready. If the debuggee also creates child processes, Windows will also send debug events that occur in all those child processes to your program as well. This behavior is usually undesirable. You can disable this behavior by specifying DEBUG_ONLY_THIS_PROCESS flag in combination of DEBUG_PROCESS flag.
- You can attach your program to a running process with DebugActiveProcess.
Wait for debugging events. After your program acquired a debuggee, the debuggee's primary thread is suspended and will continue to be suspended until your program calls WaitForDebugEvent. This function works like other WaitForXXX functions, ie. it blocks the calling thread until the waited-for event occurs. In this case, it waits for debug events to be sent by Windows. Let's see its definition:
WaitForDebugEvent proto lpDebugEvent:DWORD, dwMilliseconds:DWORD

lpDebugEvent is the address of a DEBUG_EVENT structure that will be filled with information about the debug event that occurs within the debuggee.

dwMilliseconds is the length of time in milliseconds this function will wait for the debug event to occur. If this period elapses and no debug event occurs, WaitForDebugEvent returns to the caller. On the other hand, if you specify INFINITE constant in this argument, the function will not return until a debug event occurs.

Now let's examine the DEBUG_EVENT structure in more detail.

DEBUG_EVENT STRUCT
   dwDebugEventCode dd ?
   dwProcessId dd ?
   dwThreadId dd ?
   u DEBUGSTRUCT <>
DEBUG_EVENT ENDS

dwDebugEventCode contains the value that specifies what type of debug event occurs. In short, there can be many types of events, your program needs to check the value in this field so it knows what type of event occurs and responds appropriately. The possible values are:

Value	Meanings
CREATE_PROCESS_DEBUG_EVENT	A process is created. This event will be sent when the debuggee process is just created (and not yet running) or when your program just attaches itself to a running process with DebugActiveProcess. This is the first event your program will receive.
EXIT_PROCESS_DEBUG_EVENT	A process exits.
CREATE_THEAD_DEBUG_EVENT	A new thread is created in the debuggee process or when your program first attaches itself to a running process. Note that you'll not receive this notification when the primary thread of the debuggee is created.
EXIT_THREAD_DEBUG_EVENT	A thread in the debuggee process exits. Your program will not receive this event for the primary thread. In short, you can think of the primary thread of the debuggee as the equivalent of the debuggee process itself. Thus, when your program sees CREATE_PROCESS_DEBUG_EVENT, it's actually the CREATE_THREAD_DEBUG_EVENT for the primary thread.
LOAD_DLL_DEBUG_EVENT	The debuggee loads a DLL. You'll receive this event when the PE loader first resolves the links to DLLs (you call CreateProcess to load the debuggee) and when the debuggee calls LoadLibrary.
UNLOAD_DLL_DEBUG_EVENT	A DLL is unloaded from the debuggee process.
EXCEPTION_DEBUG_EVENT	An exception occurs in the debuggee process. Important: This event will occur once just before the debuggee starts executing its first instruction. The exception is actually a debug break (int 3h). When you want to resume the debuggee, call ContinueDebugEvent with DBG_CONTINUE flag. Don't use DBG_EXCEPTION_NOT_HANDLED flag else the debuggee will refuse to run under NT (on Win98, it works fine).
OUTPUT_DEBUG_STRING_EVENT	This event is generated when the debuggee calls DebugOutputString function to send a message string to your program.
RIP_EVENT	System debugging error occurs

dwProcessId and dwThreadId are the process and thread Ids of the process that the debug event occurs. You can use these values as identifiers of the process/thread you're interested in. Remember that if you use CreateProcess to load the debuggee, you also get the process and thread IDs of the debuggee in the PROCESS_INFO structure. You can use these values to differentiate between the debug events occurring in the debuggee and its child processes (in case you didn't specify DEBUG_ONLY_THIS_PROCESS flag).

u is a union that contains more information about the debug event. It can be one of the following structures depending on the value of dwDebugEventCode above.

value in dwDebugEventCode	Interpretation of u
CREATE_PROCESS_DEBUG_EVENT	A CREATE_PROCESS_DEBUG_INFO structure named CreateProcessInfo
EXIT_PROCESS_DEBUG_EVENT	An EXIT_PROCESS_DEBUG_INFO structure named ExitProcess
CREATE_THREAD_DEBUG_EVENT	A CREATE_THREAD_DEBUG_INFO structure named CreateThread
EXIT_THREAD_DEBUG_EVENT	An EXIT_THREAD_DEBUG_EVENT structure named ExitThread
LOAD_DLL_DEBUG_EVENT	A LOAD_DLL_DEBUG_INFO structure named LoadDll
UNLOAD_DLL_DEBUG_EVENT	An UNLOAD_DLL_DEBUG_INFO structure named UnloadDll
EXCEPTION_DEBUG_EVENT	An EXCEPTION_DEBUG_INFO structure named Exception
OUTPUT_DEBUG_STRING_EVENT	An OUTPUT_DEBUG_STRING_INFO structure named DebugString
RIP_EVENT	A RIP_INFO structure named RipInfo

I won't go into detail about all those structures in this tutorial, only the CREATE_PROCESS_DEBUG_INFO structure will be covered here.
Assuming that our program calls WaitForDebugEvent and it returns. The first thing we should do is to examine the value in dwDebugEventCode to see which type of debug event occured in the debuggee process. For example, if the value in dwDebugEventCode is CREATE_PROCESS_DEBUG_EVENT, you can interpret the member in u as CreateProcessInfo and access it with u.CreateProcessInfo.

Do whatever your program want to do in response to the debug event. When WaitForDebugEvent returns, it means a debug event just occurred in the debuggee process or a timeout occurs. Your program needs to examine the value in dwDebugEventCode in order to react to the event appropriately. In this regard, it's like processing Windows messages: you choose to handle some and ignore some.
Let the debuggee continues execution. When a debug event occurs, Windows suspends the debuggee. When you're finished with the event handling, you need to kick the debuggee into moving again. You do this by calling ContinueDebugEvent function.
ContinueDebugEvent proto dwProcessId:DWORD, dwThreadId:DWORD, dwContinueStatus:DWORD

This function resumes the thread that was previously suspended because a debug event occurred.
dwProcessId and dwThreadId are the process and thread IDs of the thread that will be resumed. You usually take these two values from the dwProcessId and dwThreadId members of the DEBUG_EVENT structure.
dwContinueStatus specifies how to continue the thread that reported the debug event. There are two possible values: DBG_CONTINUE and DBG_EXCEPTION_NOT_HANDLED. For all other debug events, those two values do the same thing: resume the thread. The exception is the EXCEPTION_DEBUG_EVENT. If the thread reports an exception debug event, it means an exception occurred in the debuggee thread. If you specify DBG_CONTINUE, the thread will ignore its own exception handling and continue with the execution. In this scenario, your program must examine and resolve the exception itself before resuming the thread with DBG_CONTINUE else the exception will occur again and again and again.... If you specify DBG_EXCEPTION_NOT_HANDLED, your program is telling Windows that it didn't handle the exception: Windows should use the default exception handler of the debuggee to handle the exception.
In conclusion, if the debug event refers to an exception in the debuggee process, you should call ContinueDebugEvent with DBG_CONTINUE flag if your program already removed the cause of exception. Otherwise, your program must call ContinueDebugEvent with DBG_EXCEPTION_NOT_HANDLED flag. Except in one case which you must always use DBG_CONTINUE flag: the first EXCEPTION_DEBUG_EVENT which has the value EXCEPTION_BREAKPOINT in the ExceptionCode member. When the debuggee is going to execute its very first instruction, your program will receive the exception debug event. It's actually a debug break (int 3h). If you respond by calling ContinueDebugEvent with DBG_EXCEPTION_NOT_HANDLED flag, Windows NT will refuse to run the debuggee (because no one cares for it). You must always use DBG_CONTINUE flag in this case to tell Windows that you want the thread to go on.
Continue this cycle in an infinite loop until the debuggee process exits. Your program must be in an infinite loop much like a message loop until the debuggee exits. The loop looks like this:
.while TRUE
    invoke WaitForDebugEvent, addr DebugEvent, INFINITE
   .break .if DebugEvent.dwDebugEventCode==EXIT_PROCESS_DEBUG_EVENT
   <Handle the debug events>
   invoke ContinueDebugEvent, DebugEvent.dwProcessId, DebugEvent.dwThreadId, DBG_EXCEPTION_NOT_HANDLED
.endw

Here's the catch: Once you start debugging a program, you just can't detach from the debuggee until it exits.

Let's summarize the steps again:

Create a process or attach your program to a running process.
Wait for debugging events
Do whatever your program want to do in response to the debug event.
Let the debuggee continues execution.
Continue this cycle in an infinite loop until the debuggee process exits

Example:

This example debugs a win32 program and shows important information such as the process handle, process Id, image base and so on.

.386
.model flat,stdcall
option casemap:none
include \masm32\include\windows.inc
include \masm32\include\kernel32.inc
include \masm32\include\comdlg32.inc
include \masm32\include\user32.inc
includelib \masm32\lib\kernel32.lib
includelib \masm32\lib\comdlg32.lib
includelib \masm32\lib\user32.lib
.data
AppName db "Win32 Debug Example no.1",0
ofn OPENFILENAME <>
FilterString db "Executable Files",0,"*.exe",0
             db "All Files",0,"*.*",0,0
ExitProc db "The debuggee exits",0
NewThread db "A new thread is created",0
EndThread db "A thread is destroyed",0
ProcessInfo db "File Handle: %lx ",0dh,0Ah
            db "Process Handle: %lx",0Dh,0Ah
            db "Thread Handle: %lx",0Dh,0Ah
            db "Image Base: %lx",0Dh,0Ah
            db "Start Address: %lx",0
.data?
buffer db 512 dup(?)
startinfo STARTUPINFO <>
pi PROCESS_INFORMATION <>
DBEvent DEBUG_EVENT <>
.code
start:
mov ofn.lStructSize,sizeof ofn
mov ofn.lpstrFilter, offset FilterString
mov ofn.lpstrFile, offset buffer
mov ofn.nMaxFile,512
mov ofn.Flags, OFN_FILEMUSTEXIST or OFN_PATHMUSTEXIST or OFN_LONGNAMES or OFN_EXPLORER or OFN_HIDEREADONLY
invoke GetOpenFileName, ADDR ofn
.if eax==TRUE
invoke GetStartupInfo,addr startinfo
invoke CreateProcess, addr buffer, NULL, NULL, NULL, FALSE, DEBUG_PROCESS+ DEBUG_ONLY_THIS_PROCESS, NULL, NULL, addr startinfo, addr pi
.while TRUE
   invoke WaitForDebugEvent, addr DBEvent, INFINITE
   .if DBEvent.dwDebugEventCode==EXIT_PROCESS_DEBUG_EVENT
       invoke MessageBox, 0, addr ExitProc, addr AppName, MB_OK+MB_ICONINFORMATION
       .break
   .elseif DBEvent.dwDebugEventCode==CREATE_PROCESS_DEBUG_EVENT
       invoke wsprintf, addr buffer, addr ProcessInfo, DBEvent.u.CreateProcessInfo.hFile, DBEvent.u.CreateProcessInfo.hProcess, DBEvent.u.CreateProcessInfo.hThread, DBEvent.u.CreateProcessInfo.lpBaseOfImage, DBEvent.u.CreateProcessInfo.lpStartAddress
       invoke MessageBox,0, addr buffer, addr AppName, MB_OK+MB_ICONINFORMATION
   .elseif DBEvent.dwDebugEventCode==EXCEPTION_DEBUG_EVENT
       .if DBEvent.u.Exception.pExceptionRecord.ExceptionCode==EXCEPTION_BREAKPOINT
          invoke ContinueDebugEvent, DBEvent.dwProcessId, DBEvent.dwThreadId, DBG_CONTINUE
         .continue
       .endif
   .elseif DBEvent.dwDebugEventCode==CREATE_THREAD_DEBUG_EVENT
       invoke MessageBox,0, addr NewThread, addr AppName, MB_OK+MB_ICONINFORMATION
   .elseif DBEvent.dwDebugEventCode==EXIT_THREAD_DEBUG_EVENT
       invoke MessageBox,0, addr EndThread, addr AppName, MB_OK+MB_ICONINFORMATION
   .endif
   invoke ContinueDebugEvent, DBEvent.dwProcessId, DBEvent.dwThreadId, DBG_EXCEPTION_NOT_HANDLED
.endw
invoke CloseHandle,pi.hProcess
invoke CloseHandle,pi.hThread
.endif
invoke ExitProcess, 0
end start

Analysis:

The program fills the OPENFILENAME structure and then calls GetOpenFileName to let the user choose a program to be debugged.

invoke GetStartupInfo,addr startinfo
invoke CreateProcess, addr buffer, NULL, NULL, NULL, FALSE, DEBUG_PROCESS+ DEBUG_ONLY_THIS_PROCESS, NULL, NULL, addr startinfo, addr pi

When the user chose one, it calls CreateProcess to load the program. It calls GetStartupInfo to fill the STARTUPINFO structure with its default values. Note that we use DEBUG_PROCESS combined with DEBUG_ONLY_THIS_PROCESS flags in order to debug only this program, not including its child processes.

.while TRUE
invoke WaitForDebugEvent, addr DBEvent, INFINITE

When the debuggee is loaded, we enter the infinite debug loop, calling WaitForDebugEvent. WaitForDebugEvent will not return until a debug event occurs in the debuggee because we specify INFINITE as its second parameter. When a debug event occurred, WaitForDebugEvent returns and DBEvent is filled with information about the debug event.

   .if DBEvent.dwDebugEventCode==EXIT_PROCESS_DEBUG_EVENT
       invoke MessageBox, 0, addr ExitProc, addr AppName, MB_OK+MB_ICONINFORMATION
       .break

We first check the value in dwDebugEventCode. If it's EXIT_PROCESS_DEBUG_EVENT, we display a message box saying "The debuggee exits" and then get out of the debug loop.

   .elseif DBEvent.dwDebugEventCode==CREATE_PROCESS_DEBUG_EVENT
       invoke wsprintf, addr buffer, addr ProcessInfo, DBEvent.u.CreateProcessInfo.hFile, DBEvent.u.CreateProcessInfo.hProcess, DBEvent.u.CreateProcessInfo.hThread, DBEvent.u.CreateProcessInfo.lpBaseOfImage, DBEvent.u.CreateProcessInfo.lpStartAddress
       invoke MessageBox,0, addr buffer, addr AppName, MB_OK+MB_ICONINFORMATION

If the value in dwDebugEventCode is CREATE_PROCESS_DEBUG_EVENT, then we display several interesting information about the debuggee in a message box. We obtain those information from u.CreateProcessInfo. CreateProcessInfo is a structure of type CREATE_PROCESS_DEBUG_INFO. You can get more info about this structure from Win32 API reference.

   .elseif DBEvent.dwDebugEventCode==EXCEPTION_DEBUG_EVENT
       .if DBEvent.u.Exception.pExceptionRecord.ExceptionCode==EXCEPTION_BREAKPOINT
          invoke ContinueDebugEvent, DBEvent.dwProcessId, DBEvent.dwThreadId, DBG_CONTINUE
         .continue
       .endif

If the value in dwDebugEventCode is EXCEPTION_DEBUG_EVENT, we must check further for the exact type of exception. It's a long line of nested structure reference but you can obtain the kind of exception from ExceptionCode member. If the value in ExceptionCode is EXCEPTION_BREAKPOINT and it occurs for the first time (or if we are sure that the debuggee has no embedded int 3h), we can safely assume that this exception occured when the debuggee was going to execute its very first instruction. When we are done with the processing, we must call ContinueDebugEvent with DBG_CONTINUE flag to let the debuggee run. Then we go back to wait for the next debug event.

   .elseif DBEvent.dwDebugEventCode==CREATE_THREAD_DEBUG_EVENT
       invoke MessageBox,0, addr NewThread, addr AppName, MB_OK+MB_ICONINFORMATION
   .elseif DBEvent.dwDebugEventCode==EXIT_THREAD_DEBUG_EVENT
       invoke MessageBox,0, addr EndThread, addr AppName, MB_OK+MB_ICONINFORMATION
   .endif

If the value in dwDebugEventCode is CREATE_THREAD_DEBUG_EVENT or EXIT_THREAD_DEBUG_EVENT, we display a message box saying so.

invoke ContinueDebugEvent, DBEvent.dwProcessId, DBEvent.dwThreadId, DBG_EXCEPTION_NOT_HANDLED
.endw

Except for the EXCEPTION_DEBUG_EVENT case above, we call ContinueDebugEvent with DBG_EXCEPTION_NOT_HANDLED flag to resume the debuggee.

invoke CloseHandle,pi.hProcess
invoke CloseHandle,pi.hThread

When the debuggee exits, we are out of the debug loop and must close both process and thread handles of the debuggee. Closing the handles doesn't mean we are killing the process/thread. It just means we don't want to use those handles to refer to the process/thread anymore.

[Iczelion's Win32 Assembly Homepage]