dig.bench

Discovering unknown rules in text-based games

70 interactive games · 21 public

API docsPaperGitHub

Leaderboard

All games are human beatable, on first attempt, as validated on external human testers. Win rate: each game's wins are averaged over its runs, then those are averaged across the tier's ten games.

About

dig.bench is a benchmark of scientific discovery.

Each of its 70 games measures whether an agent can experiment to discover that game's own unknown rules. Every game is text-based, which puts it in the natural domain of language models: no visual confounds stand between a model and the discovery, so what dig.bench tests is discovery alone. Humans and frontier models play the same games with access to the same information, and progress is scored by whether the game can be beaten within a limited number of steps.

The games come in 7 tiers, depending on their difficulty. No game is easy and they all require effortful play, but humans can make the discoveries necessary to solve even our hardest games, while the best models struggle to beat games in the top tier.

Scale
70 new interactive games (21 publicly released).
What qualities of models do we test
To beat each game an agent must discover the unknown rules and apply them to solve challenges.
Evaluation
Humans and frontier models play through the same interface: identical game states, identical action sets, identical step budgets.
Schematic of a dig.bench game. A game server holding the interactive environment serves the game interface over an API, and serves models over API or MCP through a model harness: a basic harness carrying next action, game states, reasoning and rolling context, or an agentic harness that adds context management and tools with a file system. Both sides see the same game, shown as JSON in the game representation: the observation, level, lives, steps remaining, status and legal actions. The interface shows the legal actions, a creative-mode toggle, the current observation, the lives, level and steps remaining, and the step-by-step history.
The platform and equivalent human and model interfaces. 1 marks the available actions; 2 creative mode, an option in some games that allows a player to enter another level where they can experiment without it counting towards the step count; 3 the current game state; 4 game statistics; and 5 action and state history.

Play

21 of the 70 games are public. Tiers get increasingly harder for models (1 = easiest, 7 = hardest).

tier 1P-1
tier 1P-2
tier 1P-3
tier 2P-4
tier 2P-5
tier 2P-6
tier 3P-7
tier 3P-8
tier 3P-9
tier 4P-10
tier 4P-11
tier 4P-12
tier 5P-13
tier 5P-14
tier 5P-15
tier 6P-16
tier 6P-17
tier 6P-18
tier 7P-19
tier 7P-20
tier 7P-21
Reproduce it
Run any model against the games through the SDK or API.
API docs
Join us
Join our community DisCo, where you can track your progress on these puzzles and hang out with like-minded folk.
Citation
DiG-bench: Discovery in Games
Ruairidh M. Battleday, Kai Sandbrink, Jimi Cullen-Drohan, Zihan Yan, Timothy Muller, Clare Maguire, Ales Kubicek, Fraser Greenlee-Scott, Sukrit Sumant, Tri Dao, Jürgen Schmidhuber, Michal Valko, Joshua Tenenbaum, Thomas L. Griffiths, Zeb Kurth-Nelson, James C.R. Whittington
@misc{battleday2026dig,
  title={DiG-bench: Discovery in Games},
  author={Ruairidh M. Battleday and Kai Sandbrink and Jimi Cullen-Drohan and Zihan Yan and Timothy Muller and Clare Maguire and Ales Kubicek and Fraser Greenlee-Scott and Sukrit Sumant and Tri Dao and Jürgen Schmidhuber and Michal Valko and Joshua Tenenbaum and Thomas L. Griffiths and Zeb Kurth-Nelson and James C.R. Whittington},
  year={2026},
  eprint={2608.12593},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2608.12593},
}