Efficient models,
trained in the open.
Empero is an independent AI research lab working on the efficiency of language models. We publish the weights, the code and the technical reports behind our work.
Our research is about efficiency: reaching the same loss in fewer training steps, reading fewer bytes per token, and fitting more capability into the memory people already have. Every report ships with the code and the results of each run.
Recent work
PageGram
Multi-Head N-gram Memory at the Read Cost of a Flat Table
Hashed n-gram tables add billions of parameters for little compute, but their heads scatter reads across flash. PageGram places every head of an n-gram in one page: two reads per token instead of sixteen, at equal or better loss.
Read the report
Identity on Demand
Multi-head Gated Value Residuals
Each attention head decides, per token, how much of the first layer's values to draw on. In a controlled 1B-token comparison, MGVR reaches the same loss in 16 to 22% fewer steps for 0.2% more parameters.
Read the reportFinal-Token Preference Optimization
Train where the failure starts
Find the position where a failure starts and train the model to prefer a coherent alternative there. About 2,000 preference pairs took repetition loops in Qwythos-9B from 6.7% to 0.0%.
Read the postSoftware you can use today
Kojo runs in the browser. Abacus, Brewery and autocode are open source and work with the model of your choice.
Kojo
Kojo searches the web, reads images and builds pages, games and art for you. Sign in with an Empero account to create keys for its OpenAI-compatible API.
# OpenAI-compatible
base_url = "https://api.kojo.sh/v1"
model = "kojo"
Abacus
A coding agent that works in your repository. It reads code, edits files and runs tests, and has a second model check anything risky before it runs. One Rust binary for macOS, Linux and Windows, with a desktop app that drives the same agent in a window.
curl -fsSL https://abacus.empero.org/install.sh | sh
Brewery
A guided agent for fine-tuning language and image models. It helps choose a base model, prepare data and safe hyperparameters, then trains, tests and publishes the result with a model card.
GitHubautocode
A coding agent in one file, with one tool and the standard library. It writes its own tools and may rewrite its own source.
pip install empero-autocode
GitHub
More open source
- fusionchatMulti-model fusion chat
- taskgenDistillation tasks in Rust
- claude-code-proxyAny endpoint for Claude Code
- runmonitorTraining run monitor
Models that run on your own hardware
Reasoning models from 2B to 35B parameters, released with quantized GGUF builds for llama.cpp, Ollama and LM Studio.
Hugging Face, October 2026. Every number is public at huggingface.co/empero-ai.
Qwen3.8 Distill
Reasoning distilled from the Qwen3.8 frontier models into compact students, with a 262K-token context and native tool calling.
Qwen3.8-27B-Ridge
A Gated-DeltaNet-aware mixed GGUF of Qwen3.8-27B. The state path stays at Q8_0 and the mixers at Q4_K, and the native MTP head stays in the file.
Qwythos
Reasoning models with a 1M-token context, trained on verified chain-of-thought traces. Qwythos-9B-v2 removed repetition loops with FTPO.
We publish what we make. Weights, code and results go out with each release, most of them under Apache-2.0, and a model you download is a file on your disk that keeps working without us.
Before we say that something works, we test it against a baseline with the same data order and several seeds, and we report the results that did not hold up next to the ones that did.
And we size our models for the hardware people actually own: a single GPU, a workstation or a laptop. The main releases come with GGUF builds on the day they ship.
Releases and writing
Scale this work with us
Long-term compute partners
We are looking for long-term compute partners to grow our research and models step by step, with a particular interest in AMD Instinct, NPUs and other cost-efficient accelerators. Paid, discounted or sponsored: we are open to whatever works for both sides.
Build open models with us
Contributors work with the people who train our models on research, data, systems, community and the business around them, remotely and on their own schedule.
