ReactOS Newsletter: Newsletter 92

Author: Z98. Link to original: (English).
Tags: FOSS, GPL, Newsletter, ReactOS Submitted by evilslon 22.05.2012. Public material.
ReactOS Newsletter: Newsletter 92 by Z98 on 2012-05-21 - Win32 Improvements - Exceptions, Interrupts, and Traps - Farewell Pageop - Build Tools - IPv6

Translations of this material:

into Russian: Выпуск новостей ReactOS № 92. Translation complete.
Submitted for translation by evilslon 22.05.2012 Published 5 years, 1 month ago.


Win32 Improvements

Deep within GDI is code designed to carry out a series of raster operations that involve applying patterns to bitmaps. For performance reasons, it is desirable to have optimized versions of this code for the various bit depths. Formally, ReactOS only generated these optimized versions for the few ternary raster operations that were used often enough to be given a common name by Microsoft. The rest were handled in a single unoptimized version. The old code generation was also difficult to work with and the issues prompted two attempts by Timo Kreuzer to fix. The first attempt failed, but with some further prodding by Rafał Harabień, Timo tried again and came up with a better way of generating optimized functions for all raster operations. Timo separated the various raster operations into categories based on whether they needed a source, destination, and/or pattern as input. For example, an inversion operation does not require a pattern. Based off of these categories, Timo could trim unused code when a function is generated. The resulting code now generates approximately 1,200 optimized raster operation functions.

Another small but important change by Timo was switching most of win32k to use the software floating point operations he had committed many years ago. The state of the floating point unit is not preserved in kernel mode so using the FPU in kernel mode is not a good idea. This issue prompted Timo's original work in adding support for floating operations in software, but it was only recently that the most common codepaths like coordinate transformations were switched over. More specialized drawing functions such as for arcs still use the FPU.

Exceptions, Interrupts, and Traps

A while back the ARM team went through and rewrote much of the assembly code that handled low level operations into C code. Their reasoning was to make this code more readable and flexible, which are fine goals, but a few omissions and bugs unfortunately crept in. Probably the most irritating one for a few testers was the debugger break-in operation not working when WinDbg was connected while trying to debug ReactOS. A more serious issue involved some code using the floating point unit disabling interrupts and then exiting without turning them back on. These were just some of the bugs that Stefan Ginsberg squashed after his hiatus from the project.

Probably the biggest hack Stefan removed involved code that allowed page faulting while interrupts are disabled. This is illegal because when the memory manager accesses paged out data from disk, it must go through the device driver stack, which uses interrupts for pretty much everything. This hack was removed after Stefan found the root cause with some help from Timo Kreuzer. Effectively, the trap handling code that triggers on page faults was not enabling interrupts before it finished unless interrupts were already disabled when the trap code started running.

A tangential issue to all this was the situation when nonpaged pool page faults occurred. This is also something that should be impossible, since the entire point of nonpaged pool is it is always resident in memory. Stefan attributes this to the incomplete initialization/mapping of nonpaged pool page table entries and directories across all processes in the old memory manager, so that while a process may believe it has page faulted while accessing nonpaged pool, it in reality is simply due to incorrect values in the page table entry. Fortunately the ARM3 memory manager does not seem to suffer from this.

A few other issues remain, some more serious than others. ReactOS had some rudimentary SMP support that, while it did not actually allow the usage of multiple processors or cores, at least allowed the OS to run on such a platform using only one CPU/Core. This also seems to have been broken after the conversion of assembly code to C, though Stefan is unsure where the issue lies. A far more serious issue is ReactOS can trash the stack when single stepping through code while debugging. This issue Stefan believes is due to the rewrite of the trap code, though he is again insure of the exact cause.

Farewell Pageop

Several years ago, I spoke with Art Yerkes about the state of the ReactOS memory manager and its issues. One of the bigger architectural problems was the use of something called pageops, a rather unique attempt to resolve concurrent page table entry accesses. When the memory manager decides to page out to disk a page, there is always the possibility that it might be interrupted or another thread may be trying to change the state/contents of that page. This could result in data corruption if access is not carefully synchronized. Pageops attempted to deal with this by keeping track of the state of the fault handler. To store this state, a pageop actually may require additional memory be allocated, which can cause an interesting negative feedback loop if the memory manager is trying to page out memory in an effort to free up memory. After much hard work and some suggestions from Alex Ionescu and Travis Geiselbrecht of NewOS (link later), Art finally came up with a much simpler and reasonable way to deal with concurrent attempts to manipulate page table entries.

Build Tools

One of the complaints that surfaced since the switchover to CMake is the relative slowness of its builds as compared to RBuild. Some of that slowness can be attributed to the fact that CMake does a better job at dependency checking than RBuild, which is obviously a good thing. Work has been ongoing to mitigate this through both contributions upstream to help CMake with precompiled header support and switching over to a new build system called ninja. Ninja was created by Evan Martin, a Chromium developer, with an emphasis on speed. Ninja build files are designed to be generated by a higher level system such as CMake, which is what the ReactOS project intends to do. Amine Khaldi had been aware of ninja for a while and worked on getting it building ReactOS. As ninja is still a very young project, it has a lot of missing functionality. The biggest hole right now is in MSVC dependency support, which is still in a development branch for ninja. So far cooperation with the various parties involved have been very good and Amine believes that the pieces are falling into place to allow the project to be able to switch over to ninja.

One clarification for people is the distinction between a build system like ninja and a build environment like CMake. Ninja files issue the actual commands to compilers and linkers to get a program built, whereas CMake generates the build files, whether they be make makefiles or ninja files. The two complement each other and are not direct analogues of the other.


The world is slowly heading towards IPv6 and while ReactOS may not be ready anytime soon, at least the project infrastructure will not be one of those holding back adoption. Pierre Schweitzer has finished configuring the various servers and systems that ReactOS' web platforms and general IT services run on to be reachable by IPv6 in preparation for IPv6 launch day on June 6, with the intervening time to be spent on testing.

© ReactOS Team.