ReactOS Newsletter: Newsletter 93

Author: Z98. Link to original: http://www.reactos.org/ru/newsletter_93.html (English).
Tags: FOSS, GPL, Newsletter, ReactOS Submitted by evilslon 19.10.2012. Public material.
ReactOS Newsletter: Newsletter 93 by Z98 on 2012-10-18 - Desktops and Windows - PSEH - Service Management - Arguments

Translations of this material:

into Russian: Выпуск новостей ReactOS № 93. Translation complete.
Submitted for translation by evilslon 19.10.2012 Published 4 years, 9 months ago.

Text

Desktops and Windows

In the Windows operating system, the desktop one sees is made up of three things, a kernel mode desktop object, a user mode desktop window, and a desktop thread. While the desktop object can be considered almost mundane, the window and thread are somewhat special. In Windows, desktop windows are created differently from regular windows and all share a single desktop thread. This thread handles messages sent to the desktop window, though users will generally only see the desktop window if the explorer shell is not running. In ReactOS, desktop windows are treated as regular windows and have their own threads, resulting in some rather unpleasant consequences. This requires that each thread have a reference back to the desktop object, which is duplicated and handed to the thread by the function that does the actual desktop creation. This results in multiple copies of handles and references to the desktop object being scattered about and there is currently no mechanism in place to go and release the handles and references when an object is destroyed. Another problem rooted in ReactOS' handling of desktop window creation is even messier. When a window is created, the WND struct that represents its message window is allocated from the heap of the current desktop. When a new desktop window is created by an existing one, what happens is the WND struct is allocated on the old desktop's heap instead of the new desktop's heap. If the old desktop object were to go away, its heap is also reclaimed by the memory manager, which then might allocate it for use by something else. But the new desktop object is still around and relying on the WND struct allocated in that chunk of memory for receiving messages. The memory corruption that can occur basically brings down the new desktop and all the associated unpleasantness.

Giannis Adamopoulos has spent a good deal of time rewriting the NtUserCreateDesktop to remedy all of the problems described above. The biggest change was to consolidate all of the desktop threads into a single thread, which eliminated the need for duplicating handles and references for desktop windows for their respective threads. Theoretically he could have built in a mechanism in the teardown of the desktop threads in the old design to attempt to clean up lingering handles and references, but unifying the desktop threads into a single thread outright eliminates the need for the duplication of handles and references. Instead, the single desktop thread belongs to no one and a mechanism for giving it temporary access to handles and references of desktop objects on demand was set up. Giannis also did more special casing of the desktop window creation to ensure that memory allocations for a new desktop window goes to that new desktop's heap, thus fixing the potential memory corruption described above. The fix is regrettably not quite in yet due to an issue handling cursors in desktop windows. The cursor information needed happens to be in user32, whereas the code responsible for handling the cursor messages are in win32k. Once Giannis works out how to get the information he needs, a rather major architectural issue in ReactOS can be put to rest.

A question that people might ask is why bother with these cleanups, as creation and destruction of desktop windows does not intuitively seem to be a common occurrence. The first response would be, these are resource leaks and it really is good programming practice to squash such issues lest an unforeseen situation causes the problems to manifest in hard to debug ways. The second is that some applications do actually make use of multiple window stations, and hence desktop windows, for security purposes. For the curious, from a hierarchical point of view the running environment people see in Windows is organized first as sessions, which hold window stations, which themselves hold desktops. Terminal service connections each have their own session, for example, and Windows services run within a different window station than the user desktop on NT6. This isolation can be useful for sandboxing purposes and is utilized by programs that need to either protect a process from a regular user, or vice versa. As such, creation and destruction of desktop windows is not nearly as uncommon as one might assume.

PSEH

The Portable Structured Exception Handling library was originally written by retired ReactOS developer KJK::Hyperion to provide structured exception handling on non-Microsoft compilers. To achieve this, KJK in many ways exploited or worked around various behaviors in GCC, an impressive accomplishment but very much at the mercy of the whims of GCC developers who might change something PSEH relied on. Timo Kreuzer began working on a new version of PSEH, the completion of which he announced last month. Briefly, Timo made various changes to allow for better optimizations and reduce the overhead of SEH support.

First, a brief aside to explain SEH and PSEH to help make sense of what the following paragraph describes. Note that the following assumes an understanding of what stacks are in relation to program functions along with a general understanding of what exceptions are for. SEH is a Microsoft supported compiler level feature, meaning support for it must either be built into a compiler or be hacked on top of one. Microsoft's own C++ compiler obviously has support for it, but GCC and Clang do not. Implementing native SEH support in a compiler is not trivial, and PSEH was the alternative, a very clever hack on top of GCC. SEH is supported via language extensions, specifically through the __try, __except, and __finally keywords. People familiar with C++ exception handling will notice the lack of a catch keyword and the addition of a finally block. The __except keyword acts as the catch equivalent and then some. In many respects, it is very much the complexity of the __except functionality that can make SEH support complicated. The __except keyword allows a programmer to specify an exception filter, which is responsible for deciding whether the exception handling block the __except keyword wraps should be executed or not. With regular C++, the catch keyword at best specifies a single exception that its respective block is supposed to handle. In SEH, the exception filter can be its own function. An additional twist is that any function specified as a filter is not directly executed. Control is actually first passed to another function that then executes the filter, whether it be a function or a block of inlined code. Each function that has SEH blocks also get a special structure allocated on the stack that holds the necessary information for all SEH blocks within that function. That information includes the addresses of the exception filters needed by SEH to actually execute them when it receives control of the program. One other point is that the exception filters and finally blocks need access to the variables of the function they are embedded in, since the entire point of exceptions is to deal with errors that crop up due to a particular function's state. Intuitively, one might expect this to be trivial as the filters and finally blocks are a form of nested functions, but this is not the case. As stated above, the filters and finally blocks are not called directly from within the function, but by an intermediary function. This level of indirection actually muddles the function stacks somewhat and must be worked around. With a compiler that supports SEH, all this infrastructure is handled behind the scenes. With something like GCC, something must force GCC to generate code and data in a way that can then be used to replicate SEH functionality, and PSEH is that something. In addition to providing definition for the SEH keywords, PSEH is also responsible for setting up the plumbing to make sure code within SEH try/except/finally blocks work correctly. Specifically, PSEH uses nested functions to implement expression filters and finally blocks. One thing to note is that neither C nor C++ has genuine native support for nested functions. GCC achieves this with an extension that PSEH relies on, so PSEH in its current form is GCC specific. Were one to seek to port it to something like Clang, one would need to find equivalent features to achieve the same results and constitutes an effective rewrite.

Many of Timo's changes for PSEH3 dealt with giving more hints to GCC to get it to generate cleanup code instead of manually specifying them in the PSEH library. A few other changes were made to make it easier for GCC to optimize the code generated to support SEH blocks. Probably the most architecturally noteworthy change was the removal of nested function trampolines. These trampolines were small bits of code that were dynamically generated and inserted onto the stack and were responsible for setting up access to the parent stack frame before passing on control to the nested function. This was PSEH2's approach to working around the muddled function stacks caused by the indirect execution of the nested functions. In PSEH3, the responsibilities of the trampoline were split. First, Timo had GCC compile the nested functions with static addresses and built a table with them, eliminating the need to calculate their runtime addresses and allowing the SEH exception handling function to simply look them up. The stack frame pointer needed by the nested function still has to be calculated dynamically however, so the nested functions are actually called twice now. The first time the nested function will return information about its own stack frame, which is used to calculate the correct offset for the parent frame. The newly calculated address is then handed to the nested function on the second call. With both use cases of the trampoline dealt with, Timo was free to remove that functionality outright. This combined with the many other optimizations and tweaks Timo has completed makes PSEH3 a major upgrade.

Service Management

The Service Control Manager (SCM) handles starting, stopping, and monitoring of services in Windows. In the NT5 family, services can attempt to lock the service manager database when they start to ensure no other service is starting at the same time. NT6 and higher fake this as support since whatever dependencies that could have complicated concurrent starts was resolved and is no longer necessary. ReactOS on the other hand only supported locking of the SCM database for creation and deletion of services. Hermès Bélusca-Maïto, one of the newest developers to join the project, implemented that locking mechanism. Now, services that attempt to lock an already locked database will receive the correct error message, telling them to back off and try later.

Another fix Hermès worked on involved how the SCM should react to systems failing. Generally the SCM can do one of four things when a service stops unexpectedly: nothing, execute another program, restart the service, or reboot. These actions can be set by the user or a service's installer, though most just rely on the default of do nothing. Certain critical services like PnP will trigger a reboot. Right now Hermès has only implemented configuration of failure case handling as other improvements are needed before the SCM can actually detect failures in the first place.

As an aside, the only failure case that theoretically can trigger an infinite loop is the reboot case, as the failure case actions are organized as a list that the SCM iterates through. If a service crashes a second time, the SCM has already moved on from attempting to restart the service in its failure case list and in theory should not attempt to do so again unless multiple restarts are part of the finite list. Reboots on the other hand reset the list, at which point the only course of action is to boot into safe mode to try and investigate the issue.

Arguments

Windows' support of spaces in paths can make life complicated for parsing paths in command line arguments. In fact, the problem can be generalized to that of having spaces separate elements of one argument instead of indicating the start of a new argument. The workaround is to surround such arguments with quotes, which are supposed to be preserved when handed to an application, and let the program figure out what it wants do with it. ReactOS was however stripping the quotes off, which would have pretty much broken every program in existence that relied on quotes to delimit arguments with spaces. Hermès traced the problem down to code inside of ReactOS' C runtime environment and dealt with the issue.

© ReactOS Team.