Build the Fastest Inference Kernel
Inference has been a limiting factor for serving LLMs at scale. Cacheon is a live competition to write the fastest GPU kernels for LLM inference. Submit a Triton or CuteDSL kernel, beat sglang at equal fidelity, and earn up to 33 TAORewards are paid in SN14 tokens. Both the SN14/TAO price and the TAO/USD exchange rates fluctuate, so fiat values will vary over time. per day.
Kernel arena
Write the fastest GPU kernel for a target in a fixed model. Crowned improvements earn ongoing reward, scaled to how much faster they are.
Triton or CuteDSL
Submit a kernel as source. Target a typed slot: an op, a fused block, or a cross-GPU collective. Anything that makes inference faster.
sglang is the line
Same model, same machine class, same prompts. Beat the incumbent at equal fidelity to prove you are actually faster.
What is Cacheon?
An arena where GPU kernels race the stock sglang baseline. Improve inference performance with your kernel, without impacting quality, and get a reward scaled to your impact. Bigger improvements mean bigger rewards.
Every registered target is judged against its own incumbent. Many kernels can hold a crown at the same time.
Submit. Evaluate. Win.
Write a kernel
Write a Triton or CuteDSL kernel for a typed slot (an op, a fused block, or a collective) in the fixed model. Verify it locally against the reference with optima verify.
Commit-reveal your bundle
Package the kernel source in a bundle and commit-reveal it, so nobody can copy your submission before it is scored.
Validator scores it
The validator swaps your kernel into a pinned sglang engine and brackets it between incumbent baseline launches, scoring throughput gated by fidelity. Clear the bar twice, independently, and your delta crowns that target.
if not kl_gate_pass or accuracy_regressed: score = 0.0 # fidelity gate: fast-but-wrong is worthless else // Same model + seed; only your target's kernel differs baseline_tput = throughput(OPTIMA_ACTIVE=0) # stock sglang cand_tput = throughput(OPTIMA_ACTIVE=1) # your kernel speedup = cand_tput / baseline_tput # bracket the candidate with an incumbent baseline before AND after (B, C, B') require speedup >= 1 + max(margin_floor, k * measured_noise) score = speedup # else NO_DECISION — retryable, never a loss # a second, independent pass must reproduce this before it crowns the target
Output tokens/sec vs. the pinned sglang baseline, measured across two launches on the same hardware.
Per-token KL divergence vs. a stock reference run must stay under the slot's calibrated threshold. See how it works.
Real-benchmark task accuracy (GSM8K + MMLU) must not regress vs. the baseline.
Throughput comes from bracketing your kernel between incumbent baseline launches, so the delta is attributable to your one target, not machine noise. Fidelity is scored separately against a pristine, candidate-free reference. Both relative to the same pinned sglang baseline on the same hardware.
From arena to production.
- –Cacheon proved the mechanism: miners beat a pinned vLLM baseline on the same model and hardware
- –Qwen2.5-72B-Instruct on 8xH200/B200, scored against a pinned vLLM baseline
- –Miners submitted full Docker containers with any serving stack
- –Correctness gate + median end-to-end speedup scoring
- –Leader/runner-up emission split (80/20)
- –Cacheon: miners submit Triton/CuteDSL kernels for typed slots (op, block, collective), swapped into a pinned sglang engine
- –Throughput scored gated by fidelity: per-token KL vs a stock reference + real-benchmark task accuracy
- –Commit-reveal submissions; a target crowns only after two independent qualification passes beat the incumbent
- –Auditable kernel source instead of opaque containers
- –Mechanism validated on real GPUs up to gpt-oss-120b
- –Winning configurations deployed as live endpoints serving real traffic
- –Agent, RAG, and multi-turn workloads with P99 latency targets
- –PMF: teams route traffic to Cacheon for the fastest production backend
- –Routing intelligence above inference providers
- –Given a workload, determine the optimal provider, model, hardware, and serving config
- –Optimization data compounds: each discovery narrows future search
