ReactOS Newsletter: Newsletter 68

Author: Z98. Link to original: (English).
Tags: arwinss, FOSS, Newsletter, open source, ReactOS, windows Submitted by evilslon 23.01.2010. Public material.
ReactOS Newsletter 68

Translations of this material:

into Russian: Выпуск новостей ReactOS №68. Translation complete.
Submitted for translation by evilslon 23.01.2010 Published 8 years, 9 months ago.


ARWINSS, Third Time's the Charm, Maybe

There has been increasing speculation on what ARWINSS is, on what it is for, and on what it does, despite several statements released by Aleksey Bragin. The speculation has reached the point where people with incomplete information are making assertions as if they were fact when often their assertions are incorrect. ARWINSS is not intended to 'replace' the current Win32 subsystem. It exists as a parallel effort to overcome the limitations in the current Win32 subsystem by applying a different design instead of duplicating the one in Windows NT. Also, ARWINSS is only related to the Win32 subsystem, which makes it only a part of the project and not representative of the operating system as a whole. So claims that ReactOS is 'resetting' or 'restarting' are at the very least highly exaggerated. Over time ARWINSS' technical merits and demerits will become apparent as work on it and the original Win32 subsystem continues, but there is nothing that precludes the two coexisting. This provides a very high level strategic view of ARWINSS and hopefully makes clear that it is not some massive reshuffling of the project and ROS' design.

As for ARWINSS itself, the implementation is based on the architecture that Wine currently uses. There are several layers in Wine, with the majority of the reusable code sitting on top of a user mode driver that abstracts away the underlying graphics system. A separate driver is required for each graphics system one wishes to use Wine on and currently there exists a X11 driver called winex11 and a work in progress Quartz driver. There did not exist a driver for the Win32 graphics system, likely because that would have been somewhat redundant in the past. ARWINSS however implements that driver, called winent, sitting as a layer between the various Win32 DLLs that Wine implements and the kernel side component of the Win32 subsystem, Win32k. Traditionally those Win32 DLLs would directly call to Win32k with syscalls, but Wine's implementation as stated above expects an intermediary layer. What's more, the user mode driver only handled interacting with the graphics system and certain other services or resources Wine's DLLs needed were provided by something called the wineserver. Fortunately, most of the functionality wineserver provided was already implemented in ReactOS anyway so Aleksey redirected the calls the DLLs made to ReactOS libraries and functions. The presence of winex11 in some of the diagrams and even in the repository has led to some wild speculation that ReactOS was somehow intending to make use of X Windows. However, as stated above, winex11 already existed and was simply carried over as part of the initial import. It is most certainly not used in ReactOS and was effectively replaced by the winent driver Aleksey wrote. Thus none of the developers working on ARWINSS had any thoughts regarding X Windows and the attention paid to X11 came as a surprise since it had nothing to do with their work.

One additional point must be made regarding the differences between ARWINSS and Wine on Unix. Because the X Windows System has always been in user mode, communicating with it required some kind of remote procedure call and for a long time X Windows offered little to help make the calls cheaper or faster. Compared to a system call between user mode and kernel mode, this was a rather expensive operation. These constraints hurt Wine's performance but on ReactOS the winent driver simply performs syscalls to the kernel mode Win32k.

Memory Management Progress

Work on Art Yerkes' cache control branch has seen tremendous progress during the last few months, with Art reimplementing and simplifying various components. One piece to be reimplemented was the balancer, which decides when pages need to be written out to disk. Another related component is the modified page writer, which Art got working after simplifying the resource locking done during paging operations. When the page manager needs to evict pages in order to make space for new data (mpw), it will try to find a page that has not been modified, or not "dirty," so that it does not need to perform disk I/O to write out that page. One tweak to help with this is to periodically write out dirty pages while the system is idle to minimize the number of dirty pages present, which is what the mpw does. However, this previously did not work on ReactOS due to a theoretically appealing but highly complex method of dealing with synchronization across memory operations.

Originally the memory management code in ReactOS was relatively simple and over time more code was added to deal with synchronization of resources and deadlock breaking or avoidance. One attempt at dealing with resource management was pageops, which represented the state of a fault handler while it processed a specific address on a specific process. Art is not aware of anyone else using pageops and the concept might well have been created by David Welch, a former ReactOS kernel lead. However, the implementation in ReactOS resulted in a great deal of duplicated code in page fault handlers due to needing to keep track of what other fault handlers might be accessing or doing. Art removed pageops from his branch by using a sentinal value to keep other code processes away from sensitive areas until they are signaled. While this is a very simplistic way of dealing with shared access, it is the most straightforward and easiest to get working. With ReactOS, working functionality has a much higher priority than theoretical optimality right now.

Another modification Art made was in how the pager decided what pages to page out and reuse when it ran out of physical memory. What previously happened was specific memory areas would end up cannibalizing themselves to fulfill requests. If the system needed more space for user pages for example, user pages would be flushed to fulfill those requests. This caused major performance complications when certain operations caused constant paging in and out as the pager cannibalized pages it had just brought into physical memory. The pager in Art's branch is much more flexible and avoids getting into such situations.

One other major change Art did in his branch was how pages and sections were mapped. Sections are another abstraction of memory and are composed of multiple pages. Previously, the only way to determine which section a page belonged to was to check the address space the section was mapped to. However, there are situations where a section is not mapped to an address space, which causes complications when trying to page out something. Art figured out a way to identify the section a page is part of by borrowing a few bits in one of the data structures used in memory management to resolve this issue.

Cumulatively, all the work done has made paging much more reliable in ReactOS, at least on Art's branch. Paging is important for dealing with situations where the OS or applications use more memory than there physically is. Previously, ROS handled this very poorly both from a performance perspective and from a data integrity perspective. Now Art has been able to safely run ROS on systems with as little as 32MB. Getting lower may not actually be possible considering the size of the code and data that must remain in memory to help do the paging in the first place, though there is likely little need to even try going lower as such a resource constrained system could do little else besides run ROS.

Ext2 Support

With his progress in getting the cache controller in shape, Art has also managed to get ReactOS to boot off of and use an ext2 partition. The installer itself does not have support for it, so Art has been essentially bootstrapping ReactOS onto the partition and then using grub to load freeldr. Besides the work done by Pierre Schweitzer on the filesystem runtime library, Art only needed to make a few additions. The simplest was stubbing oplocks, functionality primarily relied on by network filesystems. No local filesystem actually needs this, but Matt Wu, the developer of the ext2 driver Art is currently using, was rather thorough in following the rules for writing filesystem drivers and called them to adhere to suggested practicies. This incidentally makes Matt's driver excellent for testing against. Two other more complex additions were required, including memory control blocks and range locking. Both deal with mapping of filesystem blocks to disk blocks. A few infrastructure changes are still needed to fully take advantage of Art and Pierre's work. As mentioned above, the installer does not support actually using ext2 partitions and the bootloader may also need some attention. Still, these serve as stepping stones to using more advanced filesystems in the future and the new FAT driver currently being developed will likely benefit from everything the two have done. Finally, it should be noted that much of the work is not in trunk but in separate branches, so it might be a while yet before a point release offers such functionality.