Introduction on LionsOS 0.3.0

Driver Virtual Machines

Mon, 19 Feb 2024 06:34:24 +1100

Background Reading #

See the manual on Github

Setting up your environment #

Follow the standard instructions at Getting and Building LionsOS

First steps: Odroid C4 #

There are several templates for different platforms in the LionsOS VMM Examples directory. You can take a look at them; but we’re going to start more-or-less from scratch for this.

For a driver VM we want to pass through everythng, and if possible use an upstream kernel. First step therefore is to install your linux system of choice on the hardware. We’re using the Odroid C4; examples assume the Ubuntu Jammy distribution from Armbian. Get that working first, on the eMMC, with a serial console.

Hardware setup

Tue, 12 Dec 2023 21:00:47 +1100

Setting up your Odroid-C4 hardware #

The example system targets the HardKernel Odroid-C4 platform as it is a fairly inexpensive, accessible, and reliable ARM single-board-computer.

The Odroid-C4 is available for purchase directly from HardKernel.

Official documentation for the Odroid-C4 is available here.

What you will need #

Required #

Hardware	Link
HardKernel Odroid-C4 SBC	https://www.hardkernel.com/shop/odroid-c4/
Power Supply	12V 2A
3.3V TTL-level UART to USB converter	https://www.hardkernel.com/shop/usb-uart-2-module-kit-copy/
SD card or eMMC	https://www.hardkernel.com/shop/16gb-emmc-module-c4-linux/

Optional #

Hardware	Link
PN532 NFC card reader	https://www.fruugoaustralia.com/pn532-nfc-rfid-v3-module-near-field-communication-support-and-android-phone-communication/p-228968185-488544821
VU7C touchscreen display	https://www.hardkernel.com/shop/odroid-vu7c-7inch-1024x600-hdmi-display-with-multi-touch/
Ethernet cable

Connecting everything together #

This section will show you how to connect all the hardware pieces you have to your Odroid-C4.

I/O

Mon, 01 Jan 0001 00:00:00 +0000

Input/Output #

The I/O done with LionsOS is all in user-space and is separated into components with a single purpose.

Device Drivers and Virtualisers #

Device drivers are intended to take the hardware-specific interface with the device and turn that into a standard interface based on its device class with the next component in the chain, the virtualiser.

Virtualisers are components that are responsible for taking a single connection with the driver and multiplexing between all the client connections.

libc

Mon, 01 Jan 0001 00:00:00 +0000

libc #

This page explains the LionsOS libc implementation and how to configure and build it using the provided Makefile snippet.

Overview #

LionsOS uses a custom libc setup based on our fork of musllibc. We also provide implementations of a POSIX subset and vendored compiler runtime helpers. This setup is compatible with the sDDF build system.

The final libc.a is built from three main sources:

musllibc (our fork): Provides core libc functionality, with system calls redirected to a general dispatcher.
Syscall implementations: Functionality found in lib/libc/posix/.
Compiler runtime helpers: Low-level arithmetic and runtime support from lib/libc/compiler_rt/.

Available Functionality #

Syscall implementation covers a subset of POSIX sufficient for many embedded applications. Below is a summary of available functionality.

Building

Mon, 01 Jan 0001 00:00:00 +0000

Building #

The firewall example currently relies on Microkit changes that are not part of a released version. If you have previously setup your machine for LionsOS before make sure to follow the instructions for acquiring the pre-release version of Microkit below.
Once the next Microkit release is out, we will pin to that instead.

Acquire source code #

git clone https://github.com/au-ts/lionsos.git
cd lionsos

Dependencies #

Run the following commands depending on your machine:

Building

Mon, 01 Jan 0001 00:00:00 +0000

Building #

Acquire source code #

git clone https://github.com/au-ts/lionsos.git
cd lionsos

Dependencies #

Run the following commands depending on your machine:

Ubuntu/Debian

sudo apt update && sudo apt install make cmake clang lld llvm device-tree-compiler unzip git qemu-system-arm python3 python3-pip
# If you see 'error: externally-managed-environment', add --break-system-packages
pip3 install sdfgen==0.26.0

macOS

# Make sure that you add the LLVM bin directory to your path.
# For example:
# echo export PATH="/opt/homebrew/Cellar/llvm/bin:$PATH" >> ~/.zshrc
# Homebrew will print out the correct path to add
brew install make dtc llvm qemu
# If you see 'error: externally-managed-environment', add --break-system-packages
pip3 install sdfgen==0.26.0

Arch

sudo pacman -Sy make clang lld dtc python3 python-pip
# If you see 'error: externally-managed-environment', add --break-system-packages
pip3 install sdfgen==0.26.0

Nix

nix develop

Acquire the Microkit SDK #

Run the following commands depending on your machine:

Integration

Mon, 01 Jan 0001 00:00:00 +0000

Integration #

Due to LionsOS being a component Operating System, part of the process of developing on top of LionsOS involves picking and integrating the various components and sub-systems that you want to be part of your final system.

This brings up a problem of how, as the system designer, do you put everything together without spending a lot of time getting familiar with the LionsOS components themselves?

In a Microkit based system, in order to get the final bootable image for your hardware, the Microkit tool expects you to produce two artefacts:

Debugging

Mon, 01 Jan 0001 00:00:00 +0000

Debugging #

The current debugging experience with LionsOS systems is quite tedious as the only options for debugging are to insert printfs or looking at the object dump if a certain protection domain faulted at an address.

While simulators such as QEMU offer debugging support with GDB, those debugging on hardware cannot attach a debugger.

We have been working to add first-class GDB support to LionsOS so you can do things like single-stepping over code, set breakpoints and read the state of a program.

Running (QEMU)

Mon, 01 Jan 0001 00:00:00 +0000

Running on QEMU #

To start QEMU, run the following command in the same place you compiled the system:

make qemu

You should see the following lines at the bottom of your output:

nfs: nfs.c:47:nfs_connect_cb: connected to nfs server
Starting async server on 0.0.0.0:80...

The IP address that the webserver starts on 0.0.0.0:80 is relative to QEMU’s internal network, and is not visible from your host computer.

Instead, for connecting outside of QEMU (e.g in your browser), you want to go to port 5555 of localhost (e.g localhost:5555 in your browser).

Profiling

Mon, 01 Jan 0001 00:00:00 +0000

Profiling #

In order to determine and analyse the performance of a LionsOS system, we need a systematic way of recording where CPU time is spent.

We achieve this with a statistical profiler, which means that we take regular samples of the Performance Monitor Unit (PMU) hardware to determine how much CPU time a particular Protection Domain (PD) is using.

Availability #

All the work relating to profiling support can be found here.

Reference system (Kitty)

Mon, 01 Jan 0001 00:00:00 +0000

libvmm API #

The virtual machine library (libvmm) provides the following interfaces.

`uintptr_t linux_setup_images(uintptr_t ram_start, uintptr_t kernel, size_t kernel_size, uintptr_t dtb_src, uintptr_t dtb_dest, size_t dtb_size, uintptr_t initrd_src, uintptr_t initrd_dest, size_t initrd_size)` #

linux_setup_images is used to copy Linux kernel, initial RAM disc and device tree into guest RAM ready for execution. The first argument ram_start is the virtual address of the guest RAM.

The memory pointed to by kernel is checked to see that it is a Linux image before it is copied into an appropriate place in the guest RAM.

Running (hardware)

Mon, 01 Jan 0001 00:00:00 +0000

Running on hardware #

The webserver has several Mac-address endpoints, that need to be known by several components. It expects to be run while connected to a network with a DHCP server that will hand out IPv4 addresses corresponding to each of the Mac addresses.

At the time of writing, the MAC addresses are:

Component	MAC
NFS	52:54:1:0:0:10
MicroPython	52:54:1:0:0:11

The webserver needs an NFSv3 server, whose address is known at build time. Its MicroPython interpreter can import python modules from the NFS filesystem. The NFSv3 server has to be set up to export to whatever IP address the NFS component is given by DHCP.

Running on hardware

Mon, 01 Jan 0001 00:00:00 +0000

Running on hardware #

To effectively test the firewall system when running on real hardware, each network interface must be connected to a distinct subnet with the firewall being the only means to achieve communication between the subnets. Hosts on each subnet must then be configured to forward non-local traffic to the firewall.

Unlike most LionsOS systems which use software defined MAC addresses, the firewall uses the MAC address of each NIC (although the software still requires knowledge of this).

Running on QEMU inside Docker

Mon, 01 Jan 0001 00:00:00 +0000

Running on QEMU inside Docker #

Although the firewall can be run on QEMU natively, to test it effectively it requires a more complicated networking backend than is typically required when using QEMU. The default QEMU network backend SLIRP implements a virtual NAT’d network with very little configurability. The better, more versatile option is to use a TAP network backend which supplies QEMU with a virtual NIC, and allows the host to configure the desired network topology. The problem with this is that configuring this network topology is very dependent on the operating system of the host, and in the case of macOS creating a TAP network interface is no longer supported, or at least not without the use of kernel extensions.

Testing

Mon, 01 Jan 0001 00:00:00 +0000

Testing the firewall #

This page describes how to test all the currently implemented functionalities of the firewall. The tests are for the QEMU platform, to be run inside the provided Docker container, however similar commands and tests can be used for a real hardware setup.

Testing the QEMU image in Docker #

The following commands reference environment variables set using the configuration script firewall_configuration.sh. If you are running the commands inside the container, these variables should already be set.

Contributing

Mon, 01 Jan 0001 00:00:00 +0000

Contributing to the firewall #

Currently the firewall example is in early stages of development, missing many of the basic features expected of a firewall. We consider this a community project and are inviting interested people to contribute to it and help improve its functionality and usability.

We have an extensive list of GitHub issues we are currently tracking with the firewall, starting from minor improvements to our implementation. Issues with the firewall example are all tagged with the firewall label. Three other labels are used to categorise the issues:

Building

Tue, 12 Dec 2023 21:00:47 +1100

Building the Kitty system #

Acquire source code #

git clone https://github.com/au-ts/lionsos.git
cd lionsos

Dependencies #

Run the following commands depending on your machine:

Ubuntu/Debian

sudo apt update && sudo apt install make cmake clang lld llvm device-tree-compiler unzip git qemu-system-arm python3 python3-pip
# If you see 'error: externally-managed-environment', add --break-system-packages
pip3 install sdfgen==0.26.0

macOS

# Make sure that you add the LLVM bin directory to your path.
# For example:
# echo export PATH="/opt/homebrew/Cellar/llvm/bin:$PATH" >> ~/.zshrc
# Homebrew will print out the correct path to add
brew install make dtc llvm qemu
# If you see 'error: externally-managed-environment', add --break-system-packages
pip3 install sdfgen==0.26.0

Arch

sudo pacman -Sy make clang lld dtc python3 python-pip
# If you see 'error: externally-managed-environment', add --break-system-packages
pip3 install sdfgen==0.26.0

Nix

nix develop

Acquire the Microkit SDK #

Run the following commands depending on your machine:

Running

Tue, 12 Dec 2023 21:00:47 +1100

Running #

The Kitty has several Mac-address endpoints, that need to be known by several components. It expects to be run while connected to a network with a DHCP server that will hand out IPv4 addresses corresponding to each of the Mac addresses.

As at the time of writing, the MAC addresses are:

Component	MAC
NFS	52:54:1:0:0:10
MicroPython	52:54:1:0:0:11

The Kitty needs an NFSv3 server, whose address is known at build time. Its MicroPython interpreter can import python modules from the NFS filesystem. The NFSv3 server has to be set up to export to whatever IP address the NFS component is given by DHCP.

Introduction on LionsOS 0.3.0

Driver Virtual Machines

Background Reading #

Setting up your environment #

First steps: Odroid C4 #

Hardware setup

Setting up your Odroid-C4 hardware #

What you will need #

Required #

Optional #

Connecting everything together #

I/O

Input/Output #

Device Drivers and Virtualisers #

libc

libc #

Overview #

Available Functionality #

Building

Building #

Acquire source code #

Dependencies #

Building

Building #

Acquire source code #

Dependencies #

Acquire the Microkit SDK #

Integration

Integration #

Debugging

Debugging #

Running (QEMU)

Running on QEMU #

Profiling

Profiling #

Availability #

Reference system (Kitty)

libvmm API #

uintptr_t linux_setup_images(uintptr_t ram_start, uintptr_t kernel, size_t kernel_size, uintptr_t dtb_src, uintptr_t dtb_dest, size_t dtb_size, uintptr_t initrd_src, uintptr_t initrd_dest, size_t initrd_size) #

Running (hardware)

Running on hardware #

Running on hardware

Running on hardware #

Running on QEMU inside Docker

Running on QEMU inside Docker #

Testing

Testing the firewall #

Testing the QEMU image in Docker #

Contributing

Contributing to the firewall #

Building

Building the Kitty system #

Acquire source code #

Dependencies #

Acquire the Microkit SDK #

Running

Running #

`uintptr_t linux_setup_images(uintptr_t ram_start, uintptr_t kernel, size_t kernel_size, uintptr_t dtb_src, uintptr_t dtb_dest, size_t dtb_size, uintptr_t initrd_src, uintptr_t initrd_dest, size_t initrd_size)` #