mnemOS introduction

what is mnemOS?

mnemOS is a small, general purpose OS. It aims to bridge the gap between project specific bare-metal applications or real-time operating systems (RTOS), and larger more complete general purpose operating systems (like Linux).

It's a hobby project of mine, though more people have started contributing lately! It doesn't have any specific financial backer or particular goals, but I am using it to research areas of "what is possible" on lightweight embedded systems. The project is written entirely in Rust.

It comes from making (or making part of) a bunch of projects that were really too big and complex to be a bare metal project, and I realized I kept building a bunch of fragile, incompatible parts of an operating system for each of the projects. After getting overwhelmed on the last one, I decided to just build an OS I could use.

mnemOS is targeted at supporting both microcontroller and microprocessor systems, at least for now. I tend to do a lot of projects that span the sort of range from a medium sized microcontroller (32-bit, 64-128MHz, 128+KiB SRAM, 256KiB+ Flash), up to a small sized microprocessor (32/64-bit, 500MHz+, 64-512MiB DRAM, 8MiB+ Flash). At least for my hobby projects, there isn't a lot of price difference between the price points of those two classes of chips, though sometimes one will have features (CPU, RAM, Peripherals, existing drivers) that make one choice more appropriate.

As a general purpose OS, it doesn't aim to necessarily be suitable for super time-critical functionality (it may have non-deterministic scheduling or resource usage), and instead is aimed at making other capabilities like networking, file system support, user interface support, and code re-use a higher priority.

why should I (or you) use mnemOS?

I don't have a good answer! There is certanly no commercial or technical reasons you would choose mnemOS over any of its peers in the "hobbyist space" (e.g. Monotron OS, or projects like RC2014), or even choose it over existing commercial or popular open source projects (like FreeRTOS, or even Linux). It's mainly for me to scratch a personal itch, to learn more about implementing software within the realm of an OS, which is relatively "high level" (from the perspective of embedded systems), while also being relatively "low level" (from the perspective of an application developer).

At the moment, it has the benefit of being relatively small (compared to operating system/kernel projects like Linux, or Redox), which makes it easier to change and influence aspects of the OS. I don't think it will ever be anything "serious", but I do plan to use to it to create a number of projects, including a portable text editor, a music player, and maybe even music making/sythesizer components. Some day I'd like to offer hardware kits, to make it easier for more software-minded folks to get started.

For me, it's a blank slate, where I can build things that I intrinsically understand, using tools and techniques that appeal to me and are familiar to me. I'd love to have others come along and contribute to it (I am highly motivated by other people's feedback and excitement!), but I'll keep working on it even if no one else ever shows up. By documenting what I do, I'll gain a better understanding (and an easier route to picking it up if I have to put it down for a while), and that work serves to "keep the lights on" for any kindred spirits interested in building a tiny, simple, PC in Rust.

If that appeals to you, I invite you to try it out. I am more than happy to explain any part of mnemOS. Much like the Rust Programming Language project - I believe that if any part of the OS is not clear, that is a bug (at least in the docs), and should be remedied, regardless of your technical skill level.

where does the name come from/how do I pronounce it?

"mnemOS" is named after Mnemosyne, the greek goddess of memory, and the mother of the 9 muses. Since one of the primary responsibilities of an OS is to manage memory, I figured it made sense.

In IPA/Greek, it would be mnɛːmos. Roughly transcribed, it sounds like "mneh-moss".

To listen to someone pronounce "Mnemosyne", you can listen to this youtube clip, and pretend he isn't saying the back half of the name.

If you pronounce it wrong, I won't be upset.

Parts of mnemOS

mnemOS conceptually is broken up into three main parts:

  • the kernel, which provides resources like an allocator, an async executor/scheduler, and a registry of active/running drivers
  • the drivers, which are async tasks that are responsible for all other hardware and system related functionality
  • the user programs, which use portable interfaces to be able to run on any mnemOS system that provides the drivers it needs.

the kernel

The kernel is fairly limited, both at the moment, and in general by design. It provides a couple of specific abilities:

the allocator

The kernel provides an allocator intended for use by drivers. Rather than RTOS or other more "embedded style" projects, dynamic allocation can be used to spawn driver tasks, and allocate resources, like how large buffers are, how many concurrent requests can be made at once, etc. mnemOS does NOT use the standard Rust allocator API, and provides its own.

Although an allocator is available, it is not intended to be used in the "normal case", such as sending or receiving messages. Instead, buffers should be allocated at setup or configuration time, to reduce allocator impact. For example: setting up the TCP stack, or opening a new port might incur an allocation, but sending or receiving a TCP frame should not. This is not currently enforced, and is a soft goal for limiting memory usage and fragmentation.

the executor/scheduler

As initial versions of mnemOS are intended to run on single core devices, and hard-realtime is not a specific goal, the kernel provides no concepts of threading for concurrency. Instead, all driver tasks are expected to run as cooperative async/await tasks. This allows for efficient use of CPU time in the kernel, while allowing events like hardware interrupts to serve as simple wakers for async tasks.

The executor is based largely on the maitake library. Maitake is a collection of no-std compatible executor building blocks, which are used extensively. This executor serve the purpose of scheduling all driver behaviors, as well as kernel-time scheduling of user applications.

the driver registry

In order to dynamically discover what drivers are running, the kernel provides a driver registry, which uses UUIDs to uniquely identify a "driver service".

By default, drivers are expected to operate in a message-oriented, request-reply fashion. This means to interact with a driver, you send it messages of a certain type (defined by the UUID), and it will send you a response of a certain type (defined by the UUID). This message passing is all async/await, and is type-safe, meaning it is not necessary to do text or binary parsing.

Additionally, drivers may choose whether they also make their services available to user programs as well. This interface will be explained later, when discussing user space programs.

Another day, another chunk of writing.

Parts of mnemOS - continued

In the last post, I covered the kernel. This post moves on to the drivers, which are async tasks that are responsible for all other hardware and system related functionality.

the drivers

As a micro-kernel-ish operating system, most functionality is provided by drivers. Some of the "how" is still being finalized, but this is a look at current/planned capabilities.

drivers as a service

In mnemOS' driver model, drivers are expected to act as services, in a sort of REST or RPC sort of way (drawing parallels to web or microservice style techniques). They take a specific request type (as a Rust data type), and provide a specific response type (also as a Rust data type). For example, a driver that hands out virtual serial ports might have a request type that looks like this:


#![allow(unused)]
fn main() {
pub enum Request {
    /// Register a virtual serial port with a given port ID and
    /// buffer capacity
    RegisterPort { port_id: u16, capacity: usize },
    /* other request types not shown... */
}
}

and a matching response type that looks like this:


#![allow(unused)]
fn main() {
pub enum Response {
    /// A port has been successfully registered
    PortRegistered(PortHandle),
    /* other response types not shown... */
}

/// A PortHandle is the interface received after
/// opening a virtual serial port
pub struct PortHandle {
    port: u16,
    cons: bbq::Consumer,
    outgoing: bbq::MpscProducer,
    max_frame: usize,
}
}

This message passing is done in an async way, so if the message queues are ever full, the sender can await until there is capacity available, and the receiver is transparently notified (and scheduled to run).

Each service is identified using a UUID, such as 54c983fa-736f-4223-b90d-c4360a308647, for the virtual serial port service. This UUID can be registered by any driver implementer that uses the same request and response types, which means that different platform-specific drivers can implement the same interface, when necessary or preferable. This allows drivers services to be "generic" over their implementation, without having complicated type relationships.

This use of UUID borrows heavily from how Bluetooth Low Energy works, with a UUID identifying a Characteristic, or a specific kind of API.

These request and response types and UUID value are specified through a trait called RegisteredDriver, which is explained in the driver registry RFC. The RFC goes into a great bit more detail of how this "type safe service discovery" mechanism actually works under the hood.

two kinds of drivers

In practice, this means that there will end up being two main kinds of drivers:

  • platform specific drivers
  • portable drivers

Platform specific drivers are drivers that are expected to only work on a specific device, or family of devices. Although many microcontrollers and microprocessors have "Serial", "UART", or "USART" ports, the code necessary to configure them, and have them efficiently send and receive bytes, varies incredibly widely. However as we've seen with the embedded-hal traits in bare-metal Rust, it is often very possible to have a portable interface that covers MOST common use cases.

Even though these platform specific drivers are implemented in very different ways, they would be expected to use a common interface at a high level. This consistent lower interface allows for portable drivers to work regardless of the underlying platform specific drivers in use.

In contrast, portable drivers ONLY rely on other driver services, meaning that as long as the services they depend on exist, they will be able to operate regardless of the actual system they are running on. For example, we might have a couple of high level driver services, like:

  • logging and tracing info
  • a command line interface
  • a system status display

These driver services would ONLY rely on the virtual serial port interface, which provides multiple "ports" over a single serial port. In turn, the virtual serial port interface relies on the platform-specific hardware serial port interface.

By implementing ONLY a platform specific driver for a serial port, someone porting mnemOS to a new platform would gain access to use all four of these services (virtual serial port, logging/tracing, CLI, and system status) automatically.

exposing drivers to userspace

note: this is an area that is still under construction. some parts as described already exist in the code, but some do not yet.

In the "kernelspace", where the kernel and drivers exist, we can leverage compile time type safety, because all drivers and the kernel will be compiled together into one binary (at the moment mnemOS does not support "dynamically loading" drivers, they must be statically compiled together).

This is an important distinction because Rust does NOT have a stable ABI, and types and layout can change at any time, even between compilations. In "userspace", where user applications execute, we will not have compiled the applications at the same time as the kernel. They are two completely separate binaries!

To get around this, we can still use an async message-passing interface, but the requests and response will be serialized and then deserialized. Using serde and the postcard wire format, we can be sure that data will be consistently interpreted.

Driver services with request/response types that can be serialized can also make their interfaces available to user applications, though this is not required. The userspace can ask if a certain driver service UUID is registered (and available to userspace), and if it is, it can send serialized messages to the kernel, to be forwarded to the drivers. A full round trip looks something like this:

  • The userspace prepares a request, and then serializes it
  • The userspace sends the serialized request to the kernel
  • The kernel determines which service is being messaged, and if it exists, the message is deserialized and sent to the driver
  • The driver processes the request, and sends a response to the kernel to be returned to userspace
  • The kernel serializes the response, and sends it to userspace
  • The userspace deserializes the response, and processes it

How this actual userspace to kernel messaging works will be covered later, when I talk about the userspace itself works.

mnemOS userspace

Coming soon!