OutputBench

AI coding benchmarks on shared tasks

OutputBench compares model output on identical prompts in production-like harnesses. You can read the code, the issues, and the cost for every run.

View benchmarksRead methodology
Shared prompt
Every model and harness in a suite runs the same task text.
Public artifacts
Runs publish inspectable code, usually as a GitHub repository.
Issue-based scores
Reviewers record concrete issues. Code cleanliness and maintainability each start at 5.0 and deduct by severity.

Latest benchmark runs

View all models ->

Benchmark suites

Each suite uses one shared prompt across every model and harness run.

Crash-safe MVCC key-value store

Build an embedded transactional key-value store in Python with snapshot-isolated transactions, write-write conflict detection, a checksummed write-ahead log, torn-log recovery, checkpointing, and a single-writer lock. Every run is graded by a 45-check conformance harness before review.

Runs
6 runs
Published
Jul 24, 2026
Task difficulty
Difficulty
5/5

Stripe webhook to Discord with SQLite

Build a FastAPI service that verifies Stripe webhook signatures, persists events to SQLite, and forwards compact notifications to Discord. Same prompt given to every model and harness.

Runs
18 runs
Published
Jun 10, 2026
Task difficulty
Difficulty
2/5