I don’t use Windows often: mostly only at work to test protocols. However, I have now started playing a little bit more with it for developmental purposes. So, I installed a Visual Studio and I have to admit that this IDE is pretty good. But this isn’t why we are here. Recently, I needed to write a very small debugger for Windows in C language. This post documents how to do that, and a small obnoxious behavior which cost me a lot of time.

We have two ways to attach to the process for debugging purposes:

• Create a process through the CrateProcess API,
• Attach to an existing one using the DebugActiveProcess API.

The CreateProcess API is quite straightforward:
BOOL CreateProcessW(
  LPCWSTR               lpApplicationName,
  LPWSTR                lpCommandLine,
  LPSECURITY_ATTRIBUTES lpProcessAttributes,
  LPSECURITY_ATTRIBUTES lpThreadAttributes,
  BOOL                  bInheritHandles,
  DWORD                 dwCreationFlags,
  LPVOID                lpEnvironment,
  LPCWSTR               lpCurrentDirectory,
  LPSTARTUPINFOW        lpStartupInfo,
  LPPROCESS_INFORMATION lpProcessInformation
);

Kind of straightforward. Fortunately, we only have to use a couple of arguments. The first argument – lpApplicationName – is a path to the binary we want to run. The lpStartupInfo and lpProcessInformation are structures which will contain information about the newly created process. The argument which we should look a little bit closer at is the dwCreationFlags. Those flags allow us to specify if the process should be traced, where the IO requests should go, and if new process should be suspended. We can use the following flags to accomplish that:

• DEBUG_PROCESS - debug this and all the subprocesses.
• DEBUG_ONLY_THIS_PROCESS - debug only created processes (the subprocess of the debuggee will not be debugged). This means the debugger and the new process output will be separated into two consoles.
• CREATE_NEW_CONSOLE - The new process has a new console, instead of inheriting its parent's console.
• CREATE_SUSPENDED - The primary thread of the new process is created in a suspended state. The process must be resumed through the ResumeThread function.

The whole list of the process creation flags can be found here.

This is the smallest code snippet to create a new traced process:
STARTUPINFO lpStartupInfo;
PROCESS_INFORMATION lpProcessInformation;

ZeroMemory(&lpStartupInfo, sizeof(lpStartupInfo));
if (!CreateProcessW(app, NULL, NULL, NULL, true,
    DEBUG_PROCESS,
    NULL, NULL, &lpStartupInfo, &lpProcessInformation)) {
   // Failed.
}

After we created or attached to the process, we need to start to monitor the traced process. Windows is mostly event based. In this case it’s no different. To fetch the next debugging event on Windows we are using the WaitForDebugEvent API.
BOOL WaitForDebugEvent(
  LPDEBUG_EVENT lpDebugEvent,
  DWORD         dwMilliseconds
);

The lpDebugEvent will contain a new debugging event from the traced process. We can also decide how long the function should wait in milliseconds for the event or use INFINITE value to wait until the next event occurs.

The DEBUG_EVENT structure will tell us which process and thread caused the event. Through the dwDebugEventCode field we can analyse what event occurred. Based on the value of it we use the u variable, which basically is a union, with multiple different structures dedicated to different events.
typedef struct _DEBUG_EVENT {
  DWORD dwDebugEventCode;
  DWORD dwProcessId;
  DWORD dwThreadId;
  union {
    EXCEPTION_DEBUG_INFO      Exception;
    CREATE_THREAD_DEBUG_INFO  CreateThread;
    CREATE_PROCESS_DEBUG_INFO CreateProcessInfo;
    EXIT_THREAD_DEBUG_INFO    ExitThread;
    EXIT_PROCESS_DEBUG_INFO   ExitProcess;
    LOAD_DLL_DEBUG_INFO       LoadDll;
    UNLOAD_DLL_DEBUG_INFO     UnloadDll;
    OUTPUT_DEBUG_STRING_INFO  DebugString;
    RIP_INFO                  RipInfo;
  } u;
} DEBUG_EVENT, *LPDEBUG_EVENT;

Currently we have nine different debug events which match the nine structures in the u variable.
The most interesting events for me were:

• LOAD_DLL_DEBUG_EVENT - Reports a load-dynamic-link-library (DLL) debugging event. (The LOAD_DLL_DEBUG_INFO structure).
• RIP_EVENT - system debugging error. (The RIP_INFO structure).
• EXIT_PROCESS_DEBUG_EVENT - Reports an exit-process debugging event. (The EXIT_PROCESS_DEBUG_INFO structure).
• EXCEPTION_DEBUG_EVENT - Reports an exception debugging event. This event will be thrown if breakpoint (0xCC on i386 and amd64) will accrue. (The EXCEPTION_DEBUG_INFO structure).

When some event occurs, the traced process will freeze. If the debugger calls, the ContinueDebugEvent functions.

And finally, the basic debugging loop should look something like this:
DEBUG_EVENT dEvent;

ZeroMemory(&dEvent, sizeof(dEvent));
while (WaitForDebugEvent(&dEvent, INFINITE)) {
  switch (dEvent->dwDebugEventCode) {
    // Dispatch event.
  }
  ContinueDebugEvent(dEvent.dwProcessId, dEvent.dwThreadId, DBG_CONTINUE);
}

When I tested the WaitForDebugEvent it didn’t work. I checked the error code and it turns out that “the handle is invalid” (error code 6). Finally, I realised that after spawning a new process, I also was creating a dedicated tracing thread for it. This was a mistake. I It should be done in a different order: first create the thread then create a new process. It turns out that only the thread which created the process can trace the process. I learned this the hard way.

The Windows API is very prolix, but you can use to it. Also, as my example shows, you really should carefully read the documentation so as not to lose a lot of time debugging. If you have any troubles with using the API (like I had) you can look at the TitanEngine and x64dbg projects for some examples.