Getting Started with Multikernel

What is Multikernel Linux?

Multikernel Linux runs multiple independent kernel instances simultaneously on the same physical machine. Each application receives its own kernel with dedicated CPUs and memory, free from interrupt interference, while device drivers and I/O processing run in a separate kernel on separate cores. Because no hypervisor is involved, every instance runs at native performance.

┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
│   Web Server    │  │    Database     │  │   ML Training   │
├─────────────────┤  ├─────────────────┤  ├─────────────────┤
│  Linux Kernel   │  │  Linux Kernel   │  │  Linux Kernel   │
│  (Web-tuned)    │  │  (I/O-optimized)│  │  (GPU-optimized)│
├─────────────────┤  ├─────────────────┤  ├─────────────────┤
│   CPU + NIC     │  │   CPU + NVMe    │  │   CPU + GPU     │
└─────────────────┘  └─────────────────┘  └─────────────────┘
                

Each workload runs inside its own kernel with a configuration tailored to that workload, fully isolated from other instances. A driver crash or kernel exploit in one instance cannot affect the others, and resources can be rebalanced between kernels at runtime through standard Linux hotplug interfaces.

1 Build the Multikernel Kernel

Clone the Linux kernel source with multikernel support and build it with CONFIG_MULTIKERNEL and CONFIG_MKTTY enabled.

git clone https://github.com/multikernel/linux.git
cd linux
cp /boot/config-$(uname -r) .config  # Start from the running kernel's config
make menuconfig  # Enable CONFIG_MULTIKERNEL=y and CONFIG_MKTTY=y
make -j$(nproc)
sudo make modules_install
sudo make install

The result is a standard Linux kernel with multikernel extensions. It boots and operates as the regular host kernel, and can additionally spawn and manage new kernel instances.

2 Load the Lazy CMA Module

Spawned kernel instances require contiguous physical memory. Lazy CMA, an out-of-tree kernel module, allocates this memory at runtime, so no boot parameters, reboots, or upfront capacity planning are required.

git clone https://github.com/multikernel/lazy_cma.git
cd lazy_cma
make
sudo insmod lazy_cma.ko

Loading the module creates /dev/lazy_cma. Kerf allocates memory pools through this interface automatically, and no further configuration is needed. Pools can be resized at runtime, are NUMA-aware, and are registered in /proc/iomem.

3 Install the Kerf Management Tool

Kerf is the command-line tool for creating, configuring, and managing multikernel instances. It handles resource allocation, kernel loading, and the full instance lifecycle.

git clone https://github.com/multikernel/kerf.git
cd kerf
pip install -e .

4 Launch Your First Instance

Initialize a resource pool, create an instance, and launch it:

# Initialize: CPUs 4-31 and an 8 GB memory pool (allocated via lazy_cma)
kerf init --cpus=4-31 --memory=8GB

# Create an instance with 4 CPUs and 2 GB of memory
kerf create web-server --cpus=4-7 --memory=2GB

# Load a kernel that boots straight into a Docker image (via DAXFS)
kerf load web-server --kernel=/boot/vmlinuz --image=nginx:latest

# Boot the instance
kerf exec web-server

The spawned kernel boots on the assigned CPUs and memory, running directly on the hardware. It has its own scheduler, its own network stack, and its own view of the devices assigned to it.

With --image, kerf builds a DAXFS image from the Docker image, and the spawned kernel mounts it directly as its root filesystem. DAXFS is an out-of-tree kernel module that kerf does not load automatically. Load it before running kerf load, but only when booting from a Docker image or sharing a host directory:

git clone https://github.com/multikernel/daxfs.git
cd daxfs
make
sudo insmod daxfs.ko

The spawned kernel also requires DAXFS support: either build it into the kernel, or pass --initrd with an initramfs that loads daxfs.ko. A conventional boot using --initrd alone does not require DAXFS.

Key Components

The multikernel stack is composed of several open-source projects:

• Multikernel Linux - Kernel patches that enable spawning and managing additional kernel instances through the kexec subsystem
• Kerf - Orchestration tool that manages kernel instances, resources, and lifecycle; it drives Lazy CMA and DAXFS directly, so no separate tooling is required
• DAXFS - Out-of-tree kernel module that provides a shared filesystem across kernel instances via direct memory access, enabling a shared container root filesystem and zero-copy data sharing
• Lazy CMA - Out-of-tree kernel module that allocates contiguous memory at runtime; memory pools for spawned kernels are resizable, NUMA-aware, and require no boot-time reservation

How It Works Under the Hood

• Kernel spawning via kexec - New kernels are launched through Linux's existing kexec mechanism, extended to run alongside the primary kernel rather than replacing it
• Resource partitioning - CPUs, memory, and devices are divided between kernels through standard hotplug interfaces
• Hardware queue sharing - Modern NICs and SSDs expose multiple hardware queues; each kernel receives exclusive access to specific queues, providing isolation at the hardware level
• Inter-kernel networking - Each spawned kernel receives a standard Linux network device backed by descriptor rings in shared memory; frames stay in place and only descriptors cross kernel boundaries, so applications use unmodified sockets
• Docker integration - Spawned kernels can boot directly into Docker images, using DAXFS to share the container root filesystem without an OS init layer