Using spaces in your TARGETLIBS path

I’ve been trying to find a solution on this annoying “sources” problems in the Build utility for ages. Thankfully, a post at NotAKernelGuy pointed a way to the solution. It’s in Russian, but the basic solution is simple:

LINKER_OPTIDATA = \\
"$(VS80COMNTOOLS)\\..\\..\\VC\\PlatformSDK\\lib\\mscoree.lib"

Replace that path by whatever you need, but the end result is the same: the library will be added to the response file, and spaces will be preserved and respected.

Part 2 of User-Mode Debugging Internals Article

I’ve almost finished setting up the remaining parts of my blog. I’ve added an About page and pretty much filled my BlogRoll with the blogs I try to read daily. Thanks to everyone that’s visited/linked here in the last few days.

I’m currently working on the “Publications” page of the blog to have a central repository with all my data. I will also duplicate it on OpenRCE, but that site requires a login, and I wanted to make sure anyone could freely access my stuff. The links aren’t live yet, but they should be within the day.

Also, Part 2 of my article should appear soon, but for those that want to beat the clock, you can download it for now directly here. If you haven’t read Part 1 first, make sure you do here.

Brief overview of what’s discussed:

  • Part 1 – Win32: DebugActiveProcess, DebugBreakProcess, DebugSetProcessKillOnExit, CheckRemoteDebuggerPresent, WaitForDebugEvent, ContinueDebugEvent, DebugActiveProcessStop.
  • Part 1 – Win32: Teb->DbgSsReservedData[0] and DBGSS_THREAD_DATA. SaveProcessHandle, SaveThreadHandle, MarkThreadHandle, MarkProcessHandle, RemoveHandles, CloseAllProcessHandles.
  • Part 2 – Native: DbgUiConnectToDbg, DbgUiDebugActiveProcess, DbgUiStopDebugging, DbgUiIssueRemoteBreakin, DbgUiRemoteBreakin, DbgUiGetThreadDebugObject, DbgUiSetThreadDebugObject, DbgUiContinue, DbgUiWaitStateChange, DbgUiConvertStateChangeStructure.
  • Part 2 – Native: Teb->DbgSsReservedData[1], DBGUI_WAIT_STATE_CHANGE, Teb->Tib.ArbitraryUserPointer, DBG_STATE, DBGKM_EXCEPTION, DBGKM_CREATE_THREAD, DBGKM_CREATE_PROCESS, DBGKM_EXIT_THREAD, DBGKM_LOAD_DLL, DBGKM_UNLOAD_DLL.

Part 3 will cover Kernel Mode and the Nt* APIs when it’s out.

WDK RTM Changes

I consistently did diffs (differential changes) between each new release of the WDK. It was interesting to follow the evolution of certain APIs and structures, as well as APIs which were added by mistake.

The latter happens because kits like the WDK are built from a master header file. Suppose it looks like this:

// begin_ntddk

//
// Process Functions
//
NTKERNELAPI
NTAPI
KeSetProcess(IN PRKPROCESS Process);

//
// Thread Functions
//
NTKERNELAPI
NTAPI
KeStartThread(IN PRKTHREAD Thread);

// end_ntddk

NTKERNELAPI
NTAPI
KeSetThreadDrmProtection(IN PRKTHREAD Thread);

What would happen in the DDK is that the KeSetProcess and KeStartThread would be exported, and by definition, “legit” to be used in drivers. Now suppose the developers add a new API in Vista, and don’t properly take a look at the DDK tags, you could end up with this:

// begin_ntddk

//
// Process Functions
//
NTKERNELAPI
NTAPI
KeSetProcess(IN PRKPROCESS Process);

NTKERNELAPI
NTAPI
KeProtectProcessForDrm(IN PRKPROCESS Process);

//
// Thread Functions
//
NTKERNELAPI
NTAPI
KeStartThread(IN PRKTHREAD Thread);

// end_ntddk

NTKERNELAPI
NTAPI
KeSetThreadDrmProtection(IN PRKTHREAD Thread);

Notice that the tags weren’t properly updated to keep the DRM/internal/undocumented function out from the DDK, so it will appear in the WDK. Of course, at the next release, someone is bound to notice and fixup the tags. So by doing cumulative diffs, I was able to get the prototypes of quite a few new APIs that didn’t make it into the final WDK. Of course, I don’t condone their use in a driver, but they’re useful for ReactOS/TinyKRNL development and to better understand some of the changes done in Vista.

One of the more memorable API sets that were added allow drivers (well, at least, were supposed to!) to modify the size of the kernel stack. Typically MmCreateKernelStack was a way to do this, but these new Ke functions give a much greater degree of control as well as give you a Callout function:

#define MAXIMUM_EXPANSION_SIZE (KERNEL_LARGE_STACK_SIZE – (PAGE_SIZE / 2))

typedef
VOID
(NTAPI *PEXPAND_STACK_CALLOUT) (
__in_opt PVOID Parameter
);

#if (NTDDI_VERSION >= NTDDI_WS03SP1)
NTKERNELAPI
NTSTATUS
KeExpandKernelStackAndCallout (
__in PEXPAND_STACK_CALLOUT Callout,
__in_opt PVOID Parameter,
__in SIZE_T Size
);
#endif

#if (NTDDI_VERSION >= NTDDI_LONGHORN)
NTKERNELAPI
NTSTATUS
KeExpandKernelStackAndCalloutEx (
__in PEXPAND_STACK_CALLOUT Callout,
__in_opt PVOID Parameter,
__in SIZE_T Size,
__in BOOLEAN Wait,
__in_opt PVOID Context
);

NTKERNELAPI
PVOID
KeAllocateCalloutStack (
__in BOOLEAN LargeStack
);

NTKERNELAPI
VOID
KeFreeCalloutStack (
__in PVOID Context
);

#endif

Now here’s the ironic thing: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/DevTest_g/hh/DevTest_g/t06_bugs_B0_77bda7e9-4f41-49e9-86db-04446dc9c7b7.xml.asp

“The driver switched stacks using a method that is not supported by the operating system. The only supported way to extend a kernel mode stack is by using KeExpandKernelStackAndCallout.”

I guess it’s either time for a WDK bug or an MSDN documentation bug to be opened!

However, perhaps the best WDK RTM change was this:

Original:

// This logic is a reasonable hack-o-rama to make BillG happy
// since his machine ran chkdsk after he installed Beta 3. Why?
// ’cause setup cracked a non-exclusive DASD handle near the
// end of setup, wrote some data, closed the handle and we
// set the verify bit … came back around and saw that other
// arbitrary activity had left the volume in a temporarily dirty
// state.
//
// Of course, the real problem is that we don’t have a journal.
//

RTM:

// This logic is a reasonable change. Why?
// ’cause setup cracked a non-exclusive DASD handle near the
// end of setup, wrote some data, closed the handle and we
// set the verify bit … came back around and saw that other
// arbitrary activity had left the volume in a temporarily dirty
// state.
//
// Of course, the real problem is that we don’t have a journal.

“hack-o-rama to make BillG happy” => “reasonable change”.

Got any more similar changes of your own? Feel free the post them!

Microsoft Tools for Power Users/Developers

I haven’t fully finished up setting the blog yet, but I wanted to blog about some useful and not-very-well-known Microsoft projects, tools and technologies.

Unfortunately, Microsoft advertises way too much to regular users, and doesn’t take advantage of the influencial student/developer/power user market segment, which is usually the most vocal Anti-Microsoft. You can throw words like “SuperFetch” at an engineering student that uses Linux all you want, it probably won’t impress him much. Now show him PowerShell connecting to his NFS shares through a Windows Server 2003 R2 machine with Subsystem for Unix Applications and he might raise an eyebrow.

I’ve had to explain NT’s subsystem design twice this week and I always get a “wow? really? NT can run POSIX?” Yup, and NT 5.2 SP1 can even run 64-bit POSIX and debug them in Visual Studio 2005. NT 5.2 is also called Windows Server 2003. It’s basically a kernel that’s 30% faster then XP’s and was used as the core for Vista. If you want a fast OS with an optimized kernel and don’t want to take part of the “Vista Experience”, you should consider it.

Anyways, without further ado (visit the links for information, I could blog entire pages about these). All these are FREE!

News/Community

Downloads:

Source Code:

I’ll add more as I remember them, there’s really a lot of great stuff at Microsoft that’s untapped to!

Also, if you have anything to add, please comment!