#The Oni Programming Language

This is the core repository for the Oni programming language, a simple, general-purpose systems programming language. At the moment, Oni compiles to C11, and thus has the capability of interacting directly with existing C libraries and to emit verbatim C code directly inside from Oni code. While other backends are on the roadmap, at the moment Oni aims to be to C what C is to assembly, a high-level C if you will.

The language is fairly new and thus you can expect breaking changes until v1.0.0 is released. We advice not using it in production for critical applications. With that said, the compiler is self-hosted so we are incentivized to not break things unless really needed.

#Features

• Simple and ergonomic language, with very little compiler "magic" and a consistent syntax.
• Statically typed with bi-directional inference. Generally speaking only top level annotations are required, but type annotations may sometimes be needed for type casting and variable declarations.
• Compiled to C, compatible with C, fast to compile, fast to run, will run pretty much anywhere where a C compiler is available.
• No implicit numeric conversions unless they fit withing them. Meaning a U8 can be stored inside a U16 or U32, but the opposite assignment would require explicit casting.
• Algebraic Data Types (ADTs). More specifically, product types (structs) and sum types (union/choice).
• Parametric polymorphism for types and functions. Polymorphic functions are typechecked only once on the declaration site, not during instantiation. Note that at the moment, polymorphic types do not support type constrains (interfaces/traits), instead, it uses concrete type method specializations, which serve a similar purpose and are simpler to use and understand.
• Support for the very useful lexically scoped defer keyword for resource de-initialization and/or cleanup code.
• A small but practical standard library, currently growing it as needed but with a focus on re-usability and composable interfaces. Learning lessons from modern languages like Zig and Rust and with a few twists of our own.
• Support for out of order type and function definitions. This, in combination with the import keyword, which only inserts a file a single time, enables building libraries without a complicated module system.
• Only emit what you use. Code for imported functions that are not used will not be generated on the backend, reducing the compilation and linking overhead.
• Automatic zeroed variables for stack resources, in line with the concept of zero-is-initialization, which is used extensively on the standard library.
• Ability to emit code directly to the backend to avoid having to deal with a more complex building pipeline. This enables us to define and use C functions directly from Oni, including the use of C's assembly capabilities, very useful to create "glue" code and interfaces.

#Building/Installation

A bootstrap file with a working version of the compiler is provided on src/oni/bootstrap.c. You can build it with your C compiler of choice or use the make or make bootstrap command. This will create a binary artifact on build/oni.0.

The bootstrap compiler can be used to build the current version of the compiler with make v1. This is the recommended approach for developing the compiler itself.

To build a binary release of the current compiler:

make release
sudo make install

This will install the compiler binary oni, a runner script oni-run and the standard library into your system. The runner script depends on the tiny C compiler tcc for compiling and running the generated C files. The performance of the compiled code by tcc is quite unoptimized, but it compiles orders of magnitude faster than clang or gcc with optimizations turned on, which makes it suitable for running .oni files as scripts.

If you want to change the installation directories of the binaries and standard library path, you can do so with make variables:

make release INSTALL_DIR_STDLIB=/my/install/path/stdlib
make install INSTALL_DIR_STDLIB=/my/install/path/stdlib INSTALL_DIR_BIN=/my/install/path/bin

You can use musl for a fully static binary build of the compiler:

make release CC=musl-gcc CFLAGS="-O2 -static -std=c11 -fno-pie -no-pie -s"

#Getting started

#Hello world

The simplest possible hello world uses std/debug to give you the fmt and errfmt output formatters for stdout and stderr respectively:

import "std/debug"
Dbg.fmt.println("hello world!")

Save this file as example.oni and run it with:

oni-run example.oni

For a more involved example, we can create the output formatter ourselves with a statically allocated buffer, this is what std/debug does under the hood. We can also add a hash-bang string to use oni programs as scripts:

#!/usr/bin/env oni-run

import "std/fmt"
import "std/io/stdout"

; Initialize the IO.Writer for stdout with a static buffer.
let buf: [1024]U8
let w = IO.StdOut.init((@buf):Ptr, 1024)
let fmt = Fmt.init(@w)

; String formatting uses method chains for substituting the magic `%` character.
fmt.println("hello % how are you doing?").str("friend")

; It accepts numbers, strings, characters and different formatting options.
let number =  0xFFAA
fmt.println("hex: % hex_padded: % int: %")
    .hex(number)
    .hex_padded(number, 8)
    .int(number)
fmt.println("%\n%  ONI  %\n%")
    .char_repeat('=', 23)
    .str_left_pad("::", '-', 8)
    .str_right_pad("::", '-', 8)
    .char_repeat('=', 23)

This can now be run as follows:

chmod +x example.oni
./example.oni

#Memory allocations

Oni utilizes manual memory management for heap allocated resources. The standard library provides a few useful memory allocators, like Arenas or libc's allocator, that uses malloc and its ilk. The standard library is designed to take an Allocator interface for functions that need to allocate memory. An allocator is simply a struct holding a Vtable of function pointers and some context data:

; file: std/allocator.oni
struct Allocator {
    malloc_fun:  (ctx: Ptr, size: Int -> Ptr)
    calloc_fun:  (ctx: Ptr, size: Int -> Ptr)
    free_fun:    (ctx: Ptr, ptr: Ptr, size: Int)
    realloc_fun: (ctx: Ptr, ptr: Ptr, old_size: Int, new_size: Int -> Ptr)
    ctx:         Ptr
}

Allocators expose the methods for malloc, calloc, free and realloc, and are relatively easy to create, using some platform API functions or other allocators as base. For example, let's use an Arena allocator using libc's allocator to initialize it, and then wrap up the arena allocator into a TracingAllocator to log all memory allocations. Finally, we can use an Array iterator to print its elements:

import "std/debug"
import "std/array"
import "std/allocators/libc"
import "std/allocators/arena"
import "std/allocators/tracing_allocator"

let arena = Arena.new(1024, LibC.allocator())
let arena_alloc = arena.allocator()
let alloc = TracingAllocator(arena_alloc, Dbg.fmt).allocator()

let arr = Array.new::(Int)(2, alloc) ; Allocates 16 bytes.
arr.push(1, alloc)                   ; Doesn't allocate, 8 bytes used
arr.push(2, alloc)                   ; Doesn't allocate, 16 bytes used
arr.push(3, alloc)                   ; Reallocates 32 bytes, 24 bytes used
arr.push(4, alloc)                   ; Doesn't allocate, 32 bytes used

foreach item in arr.iter() {
    Dbg.fmt.println("i: %").int(item)
}

This is a bit of a convoluted example, but you can see how this approach can be quite useful in many circumstances. For example, we could create an arena with a statically allocated buffer instead of using malloc. That arena's allocator could then be used with any stdlib function that requires it or subdivide it's memory as needed. Generally, it is recommended to group all allocations with a given lifetime together and free them in bulk when done. Arenas have also the capability of just resetting it's size to zero with the Arena.reset method. This is very fast and useful for temporary allocations, like those in a given graphics or audio frames on real time applications.

#Calling C libraries

In addition to the standard library and compiler, Oni comes bundled in with bindings to some useful external libraries, like Raylib on the src/binding directory. These files are licensed as their original libraries and can be considered an extra nice to have thing, but due to their external nature we can't possibly commit to have them be fully updated with the same priority as the rest of the project.

Here is a minimal hello world using Raylib:

import "bindings/raylib"

let title = "hello world :: oni :: raylib"
let win_width = 800
let win_height = 600

; Colors.
let bg_color = RL.Color(0x1a:U8, 0x1d:U8, 0x23:U8, 255:U8)
let fg_color = RL.Color(0x72:U8, 0xde:U8, 0xc2:U8, 255:U8)

RL.init_window(win_width, win_height, title.mem)
while not RL.window_should_close() {
    let space_pressed = false
    if RL.is_key_down(RL.Keys.SPACE) {
        set space_pressed = true
    }
    let msg = "Press space plz"
    RL.begin_drawing()
    defer RL.end_drawing()
    RL.clear_background(bg_color)
    let x_pos = 300
    let y_pos = 250
    let font_size = 20
    if not space_pressed
        then RL.draw_text(msg.mem, x_pos, y_pos, font_size, fg_color)
        else RL.draw_text("yay you did it!".mem, x_pos, y_pos, font_size, fg_color)
}

Since we need to link with raylib, it would need to be installed as a system library. We can pass CFLAGS to the C compiler used by oni-run as follows:

CFLAGS=-lraylib oni-run example.oni

This example showcases some additional Oni features, like namespaces, external function aliases (begin_drawing instead of Raylib's BeginDrawing) and how Oni strings can be used directly on C programs by just getting their memory location. This is possible because Oni and it's standard library ensures new strings created with those functions are null terminated, but we need to be careful with these kind of operations if we are manually modifying Str objects.

#Injecting C code and creating bindings on the fly

Oni has available a number of compiler directives that enables the emission of raw text into the backend output. This can be used to create C bindings directly from within an Oni program. For example, we can use C stdlib to print hello world by emitting the C code necessary to the top of the global scope and binding an external function with the same type signature:

; Create a custom binding for a hello world C function.
#global
    `#include <stdio.h>
    `#include <stdlib.h>
    `void hello(int64_t x) { printf("%d: hello world from C!\n", x); }
fun hello(x: Int)

; Run it from Oni.
hello(42)

#Calling Oni code from C

We can easily expose Oni functions as external symbols and link them together with existing C programs. To do so, make sure the functions are marked as public with the pub keyword to avoid name mangling and to ensure code is generated for them even when unused. Additionally we can use the as keyword to specify the external symbol name:

; file: arith.oni
pub fun add(x: S32, y: S32 -> S32) = x + y
pub fun sub(x: S32, y: S32 -> S32) as substraction = x - y

; file: example.c
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

extern int32_t add(int32_t, int32_t);
extern int32_t substraction(int32_t, int32_t);

int
main(int argc, char *argv[]) {
    printf("1 + 2 = %d, 1 - 2 = %d\n", add(1, 2), substraction(1, 2));
    return 0;
}

Since we are using Oni code as a library, we need to ask the compiler to not emit a main function:

oni arith.oni --no-main -o arith.c
cc example.c arith.c -o arith
./arith

Note that if we have any global variables, assertions or main like behaviour we need to ensure the Oni initialization function runs before calling those functions. By default this is the __init function, but could be renamed with the --entry <name> compiler option. Let's look at an example of this, with the following files:

; file: hello.oni
import "std/debug"
pub fun say_hi() = Dbg.fmt.println("hello there!")
Dbg.fmt.println("Oni initialization done!")

; file: example.c
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

extern void say_hi(void);
extern void oni_init(void);

int
main(int argc, char *argv[]) {
    oni_init();
    say_hi();
    say_hi();
    return 0;
}

Compile it and run it as follows:

oni hello.oni --no-main --entry oni_init -o hello.c
cc example.c hello.c -o hello
./hello

#Overriding the standard library

The Oni compiler can embed a default path to the standard library, which can't be changed since it's built into the binary. However, this has the lowest priority on the search path, so it's easy to override by just adding an include path directory of our choosing. This can come handy in cases where we are compiling for a platform without a libc implementation and we are trying to call some Oni stdlib function that requires it, like on some game consoles or embedded systems.

For example, the panic function prints a message to stderr and aborts the program. But what if we don't have access to fprintf but want to use std/array, which depends on std/panic? Well, we can just create our own project specific implementation of std/panic, with a directory structure like this:

.
└── src
    ├── main.oni
    └── std
        └── panic.oni

Where src/std/panic.oni could just be an infinite loop to halt the program or use some other way to log the error message:

; file: src/std/panic.oni
fun panic(msg: Str) = while true { nil }
; file src/main.oni
import "std/array"
...

Making sure we run/compile including the src directory on the search path, which we may want to do anyway to have an easier time with the project organization:

oni-run -I src/ src/main.oni

#Tests and examples

To run the test suite with the v1 compiler, use:

make tests CC=tcc

The test files are sort and do benefit from the faster compilation times of tcc but feel free to use any C compiler and options you require.

To perform the "triple check", where an initial version of a compiler is used to compile itself three times and validate that the generated code across versions is identical:

make check-three-way CC=tcc

There are also a few example projects available in their respective directories of the example folder. To compile them into build/examples use:

make examples

Note that some of them may require you to have Raylib installed as a system library. Feel free to modify the compiler options as needed for your particular machine.

#Issues

At the moment, all project issues are being tracked on the TODO.md file at the root of this repository. If you encounter any issues, feel free to send an email to the mailing list to start a discussion. As the project grows we will investigate other approaches for issue tracking, but this is a fairly low friction solution for me at the moment.

#Documentation

A language reference that runs over the basic syntax and semantics can be found in this repository. The standard library documentation will be provided separately. At the moment is small and perhaps it's more clear to just read the code directly on the src/std folder. In the future, it would be helpful to have some automatic tooling to generate docs from comment strings.

#License

This project is licenced under the Hippocratic License terms. In short, this allows use of this software and derived work to parties not involved in harming or discriminating against people on any basis, being fair to your colleagues and employees, and not using it for military or law enforcement purposes. Additionally, you shall treat this software and derivative work as copyleft, meaning any modifications in re-distribution should be treated with these same terms. Within these conditions, you are free to use this software for commercial purposes.

If for whatever reason you need a different license for this software, you can contact me and we can discuss some other licensing terms as long as they are reasonably within the parameters of the existing license.

#Contributing

This programming language was created with a strong vision for keeping it ergonomic but at the same time as simple as possible. Contributions are welcome, but contributors must keep in mind the original vision for this language.

Ideally we should try to remove features when not needing them, and adding them just when they feel essential. Instead, the most valuable contributions would have to do with the standard library, bugfixes and documentation. Before implementing major pieces of work for this project, please reach out to discuss them first instead of jumping right into the implementation. This would ensure a smooth process and avoid wasting or duplicating effort.

Aside from this repository, other ways of contributing include using and testing the language in your own projects and talking about it with other folks.

Note that contributions created partly or fully with LLM/AI or other generative technologies built on top of theft and the abuse of power won't be accepted and you will be permanently banned from any further contributions. Similarly, any instance of discrimination or harassment on any basis in this or other communities will not be tolerated. Let's treat each other with kindness and trying to be constructive in our feedback and discussions.

#Support

This is a passion project and work will continue regardless of financial support, however, donations are appreciated if you feel inclined. You can find some ways of doing so in this page. In any case, you can put that money towards people that need it way more than me and I encourage you to do so. There is way too much war and way too many folks suffering from hunger to make a list here, so please just find a charity you trust and drop a few bucks on them.