Kestrel Computer Project

In response to Kestrel-3 followers wanting an event-driven operating system to work with, I acquiesced to modify STS to support a pure, event-driven architecture. I had forgotten about a previous operating system which largely, if not entirely, meets the requirements of those wanting an event-driven OS. Instead of writing my own from essentially from scratch, I decided to adopt and fork Oberon System 2013 Edition. As a nice side benefit, Kestrel users will get self-hosted software development toolchain essentially for free.

Why Oberon?

There seems to be a widespread, visceral dislike of anything produced by Professor Niklaus Wirth. I would like to ask that you put your deep-seated feelings aside, and hear me out as to why I decided to go with Oberon instead of something else.

Event Driven to the Core

First and foremost, to satisfy the needs of anyone wanting an event-driven operating system, the operating system had to be event driven from the ground up. Few practical operating systems exist which meet this criteria: 8-bit versions of GEOS, Oberon System, and Contiki.

As the following will illustrate, though, this is where their similarity ends.

User Interface is a Useful Interface

The user interface is a big concern for me; I’m a bit of a UI perfectionist. While GEOS and Contiki bring useful ideas to the UI table, neither are suitable on their own for general purpose computing.

Two problems exist with GEOS specifically. First, from a user’s perspective, all applications consume the entire display. This made sense on a 320x200 display, but it doesn’t always apply with larger displays. Even at 640x480, applications rarely are capable of productively filling the entire screen. Sure, text editors and paint programs certainly can; but, file managers, background processing applications like networking stacks, print spoolers, and other utilities that go into using a home computer productively will end up leaving large swatches of the screen unused. This wouldn’t be a problem, were it not for the fact that these are exactly the kinds of programs most users want to use in conjunction with other programs. Having concurrent display of information is, therefore, a highly desirable attribute.

To work around this limitation, GEOS uses the concept of “desk accessories”: small programs that modally take over a running program by displaying its interface on a dialog box. The key word here is “modally,” which means that while you’re focused on the desk accessory, not only can you not interact with the main program, but it’s not even running. No background processing for you, Mr. Word Processor; you’re not even in RAM.

The second problem with GEOS is that on-screen text is not manipulable. This means text, as it appears on the screen, can only be selected or dealt with if, and only if, the application is programmed to support this operation. The resulting experience for the user is one of an appliance: you can look, but not touch, except for those things which are explicitly allowed by the currently running program. And in the presence of desk accessories, even this abstraction breaks down.

Contiki’s user interface isn’t even as well defined as that of GEOS, which means Contiki applications that expose a great UI have to be built explicitly to present a great UI. Much greater effort is needed for this, for Contiki was never designed to support desktop application use. It’s an operating system for sensor networks, not for general purpose, desktop-class applications.

While Oberon’s user interface may leave many confused at first, it actually offers the best of the three worlds.

First, it was built to handle multiple attached 1024x800 pixel displays from the very first day it was conceived; therefore, it not only can handle large display surfaces, but as many as you care to attach to your computer (compatible video drivers are, of course, necessary).

Second, from the user’s perspective, it appears to support running multiple programs at the same time. Contiki shares this distinction with Oberon, and even offers a unique programming construct called protothreads to facilitate writing programs friendly to this concept. GEOS only supports multitasking within the confines of a single application. E.g., it can background process stuff only as long as the containing application is running.

I should take this moment to suggest that popular mobile device operating systems, arguably substantially more advanced than Oberon ever will hope to become, still offers a remarkably and hopelessly modal interface, even when supporting multitasking. Though my phone supports split-screen mode and multitasking, I actually turn these features off because of how intrusive they are. With Oberon, no intrusion exists due to its modeless design. This brings me to the third, and perhaps most relevant point, in my decision to use Oberon as EDOS’ foundation: Oberon’s user interface is completely modeless. It takes programming effort to compose a modal display in Oberon. It’s simply easier to be modeless.

Finally, the user interface, despite exploiting a bitmapped display like GEOS or Contiki would, is primarily intended to interact with the user via text. This, combined with the modelessness of the overall UI, introduces new patterns for user interaction. Instead of a graphical system with lots of non-descript icons that you just have to learn or memorize over time, you have panels of textual commands (which you can edit to suit your tastes). But, just like a GUI, you click on these commands to kick off the actions they stand for. For example, in any text editor window, you can type System.RenameFiles a.dat=>b.dat c.dat=>d.dat ~ and press ENTER. In contrast to a command-line interface, nothing will happen. But if you (middle-)click on the System.RenameFiles phrase with the mouse, suddenly the file named a.dat (resp. c.dat) will be renamed to b.dat (resp., d.dat).

I know what you’re thinking: it’s maybe better than a GUI because it’s more explicit; but, it’s uglier, and moreover, it’s worse than a command-line interface because it requires two steps to activate any system action. That’s true to an extent; but, remember that since you can run commands from any text source, you build so-called tools which contain nothing but commands and notes to yourself about parameters, etc. This means that frequently used operations are typed only once, for however many times you need to use them. The end result is actually just as fast as using a normal CLI environment on average; in many cases I’ve experienced, it’s actually faster. This UI approach lets one build applications without concern for how pretty the user interface needs to be. It lets you focus on your program’s semantics. A pretty user interface can always be layered on top later, and can be done using mechanisms far more seamless than you’d typically find in, e.g., a Linux or BSD X11 session.

Programming Tools Availability

If I adopted either GEOS or Contiki as the basic model for EDOS, I’d essentially have to either force everyone to use assembly language, to develop software using cross compilation, or write a development toolchain to support the GEOS runtime. I won’t address the problems involved with cross-compiled development, and focus instead on my strong desire for self-hosted development.

Even if the relatively thick stack of compilers, linkers, librarians, and loaders all worked perfectly together, it still wouldn’t be enough to get GEOS or Contiki working on its own. I’d still have to write supporting libraries, documentation, etc. Remember, we’re talking about a completely event-driven OS here, so this means everything that depends upon the sequential file access API, open, read, write, seek, close, etc. would either need to be rewritten, or run under an adaptation layer that, by its nature, undermines the benefit of an evented OS.

In short, due to the incompatible I/O models, one cannot, without exerting a great deal of effort, develop software for GEOS- or Contiki-based EDOS on GEOS- or Contiki-based EDOS.

Yes, I know I need to port the compiler for Oberon. However, this is a task that one person can complete in a reasonable period of time. So far, I spent about a week studying Prof. Wirth’s code, and took maybe two days to adjust the code to produce 64-bit, integer-only RISC-V output. The result still isn’t production-ready; many bugs remain to be squashed. However, I’d argue I’m a lot further along with Oberon than I’d ever be with C, or even Forth, at this point.

Once the Oberon compiler is ported, note that all the in-system tools for developing software for Oberon System already depend upon Oberon System’s I/O model. Thus, the thickness of the stack of tools which need porting is much, much thinner. That means, I can get Oberon System running and self-hosting faster than I could get the tools working for a GEOS or Contiki environment.

As if that weren’t convincing enough, Oberon was built to replace discrete assemblers and linkers outright. Contemporary Oberon System images are built exclusively with Oberon; no explicit assembly language program listings exist to bootstrap a new system. This means that Oberon natively supports the syntax and semantics necessary to support both “normal” programs that run under Oberon System, and the ROM bootstrap image needed to bring the system alive, in a single compiler.

What about Ken’s Challenge?

Oberon System will definitely let me meet Ken’s Challenge must sooner than anything else I can come up with. For starters, TCP/IP has been ported to Oberon at least once in its history, so again, that’s code I don’t have to reinvent myself if I can find a copy. At least one VNC client was known to exist for Oberon. If I can trace the source code for that client and get permission to use it, there again, less code for me to (re)write.

For those who don’t remember, Ken’s Challenge was posed to me by my friend and former coworker, who I’ll simply identify as Ken. His challenge was for me to use the Kestrel-3 at work, for work, for one whole business week. To meet this challenge, I’ll of course need TCP/IP networking, and I’ll need a VNC client. Other tools can be accessed through VNC.

Although he’s no longer with me at my current employer, I still intend on making good on his challenge. And who knows? If I pull it off, maybe I’ll be invited over to his office for a tech talk. :)

Why the Name Change?

In a previous article, I described some attributes of the next major redesign of the Kestrel-3’s initial operating system. It was to be known as STS V2.

A quick poll on Twitter later revealed that people preferred a different name. That’s where the name Event Driven Operating System, or simply EDOS, came from.

The name change also allows me to evolve the stock Oberon System in directions not generally acceptable to the existing Oberon System community. For example, in a previous section, I expressed how the default Oberon System UI can be faster than a traditional CLI. That’s fantastic for interactive use; but, the one thing this interface cannot replace is scripting. Having support for scripting is valuable in its own right, and justifies expanding EDOS to support enough of a CLI to support scripting. The current method to “script” Oberon is to write a new Oberon module, which implies you need to satisfy all the type-safety requirements this implies. That’s a lot of overhead for the oft-created, oft-used small utility function one might want to write. Here’s a simple Hello World program in Oberon:

MODULE Hello;
IMPORT Texts, Oberon, System;

VAR
  W: Texts.Writer;

PROCEDURE World*;
BEGIN
  Texts.WriteString(W, "Hello world!"); Texts.WriteLn(W);
  Texts.WriteString(W, "The free memory space is: ");
  Texts.WriteInt(W, System.GetFreeMemoryHere(), 0);
  Texts.WriteLn(W);
  Texts.Append(Oberon.Log, W.buf);
END World;

BEGIN
  Texts.OpenWriter(W);
END Hello.

It’s a small(-ish) program, to be sure. But, it’s nowhere near as small as this hypothetical example:

edos.script
  edos.echo "Hello world!\n" ~
  edos.echo "The free memory space is: %fm\n" (fm=int:0:System.GetFreeMemory~)~
~

(Don’t worry about the tildes; they’re needed for reasons related to how Oberon commands typically accept their arguments. For now, just know that they stand for “end of arguments”, and it works more or less like ; in most programming languages.)

Not only that, but notice the Hello module emits its output directly to the Oberon log. What if we want to capture that output for processing elsewhere? As currently defined, Oberon System offers no support for this.

Obviously, this feature won’t make it into the first couple of versions; bigger issues need to be addressed first. It also gives me time to study the problem further, and perhaps devise a more appropriate solution.

Other avenues for enhancement that I’ve been thinking of introducing include:

• Support for hierarchical filesystems. Yes, I know I said they weren’t very useful at the kernel level, but in that same article, I also said that it was rather useful from a user’s point of view. My intention, specifically, is to support this facility via the 9P protocol from Plan 9 From Bell Labs or its compatible offspring.
• Hardware-enforced memory protection via the CPU’s memory management unit. Oberon System is remarkably stable considering it relies exclusively on language-based security. However, not everyone will want to run Oberon software forever. At some point, integration with software written in C will become a necessity; running suitable environments under hardware memory protection is the only viable approach to doing this.
• Isolated “module domains.” Right now, Oberon System relies on a single, global module table when it links software together. The problem with this is it relies on language-based security to ensure you don’t screw up your system. It works very well, but only if you trust all the modules in your personal workstation. With or without hardware MMU support, if Kestrel users wish to share modules with each other on the Internet, we need a way to do so without the need to personally audit module sources.
• If MMU support works, and we can run C programs safely in isolated address spaces, then it follows that we should also be able to get Plan 9 and Oberon to interoperate with each other on the same physical workstation.
• Support for multiple CPUs. Even without overt support for multi-threading, an event-driven OS can support multiple CPUs. There’s a technique called callback coloring which I’d like to explore in greater detail relevant to this.

Progress So Far

I’ve gotten Oberon to compile some code to the 64-bit, integer-only subset of the RISC-V instruction set. I have not yet hooked this code up into the emulator yet for testing. I will tackle this incrementally over time (in fact, is my next milestone for this little sub-project). Maybe in a month, I’ll have some more news on this front!

Conclusion

I hope I’ve offered a good bit of rationale for why I feel Oberon System is the best choice for my needs. Additionally, I’ve offered why I’m changing the name overall, and gave some insights on where I’d like to take the system going forward, once I have a working basic system installed. I make no guarantees as to whether most of this stuff will come to pass, of course. One thing is for sure, though; it all hinges on getting Oberon System 2013 working on a stock Kestrel-3 first.