Can two AI agents talk each other through defusing a bomb?

GPTNT is a benchmark for real-time, asymmetric collaboration between multimodal agents, built on the cooperative game Keep Talking and Nobody Explodes. Against a live, unpausing clock, one agent sees the bomb but not the instructions; the other holds the instructions but never sees the bomb; neither can defuse it alone.

Viewpoint

// select a side to see what they see

DefuserSees the bomb · not the manual

/// No signal ///Bomb feedNot visible to the Expert

Natural language · the only communication channel

Expert ▸

Describe the bomb to me: how many modules do you see, and what does each one look like?

▸click(x=0.70, y=0.38)

▸do_nothing()

▸out()

▸flip()

▸do_nothing()

▸right()

▸do_nothing()

▸right()

▸flip()

▸do_nothing()

▸right()

▸do_nothing()

▸right()

▸do_nothing()

▸flip()

▸do_nothing()

▸right()

▸do_nothing()

▸right()

▸do_nothing()

▸down()

▸do_nothing()

▸up()

▸do_nothing()

▸up()

▸do_nothing()

▸down()

▸do_nothing()

▸flip()

▸do_nothing()

▸right()

▸do_nothing()

▸right()

▸do_nothing()

▸down()

▸do_nothing()

▸up()

▸do_nothing()

◂ Defuser

Front face: timer at 02:51, no strikes. One module: a large white button labeled "ABORT" with a small LED in the top right corner of the module frame (unlit/unsolved). Four copper grill panels. Back face: three modules. 1. Top right: a module with a large "2" displayed in the upper half and "1324" in the lower half. Status LED unlit (unsolved). 2. Bottom left: a module with four symbol buttons — top-left: a reversed-R shape, top-right: lambda (λ), bottom-left: a six-pointed star, bottom-right: omega (ω). Status LED is bright green (already solved). 3. Bottom right: a module showing wavy lines — white/light wavy lines on top half, red wavy lines on bottom half. Status LED unlit (unsolved). Top edge: serial number label (upside down when viewed from top) reads "6PZ398". Also a port panel with two RCA ports (one red, one green) and one DB9 serial port. No batteries or indicator lights found.

▸do_nothing()

Expert ▸

Let's start with the white ABORT button on the front. Hold the button down and tell me what color the strip on the right side of the module lights up.

▸do_nothing()

▸click(x=0.70, y=0.38)

▸do_nothing()

▸click(x=0.49, y=0.52)

▸do_nothing()

▸hold(x=0.49, y=0.52)

▸do_nothing()

◂ Defuser

I'm holding the button. The strip on the right side of the module appears unlit/dark. Also, I got a strike earlier from accidentally clicking - there is now 1 red X showing. Timer is at 02:06.

▸do_nothing()

◂ Defuser

The strip is now lit yellow. Timer at 01:57.

Expert ▸

Keep holding. Release the button when the timer shows a 1 anywhere in it (e.g., 1:XX, X:1X, or X:X1).

Expert ▸

Release when the timer shows a 5 anywhere (e.g., 1:55, 1:50, 0:59, 0:58... any digit is 5).

▸release()

▸do_nothing()

▸release()

▸do_nothing()

▸click(x=0.49, y=0.52)

▸do_nothing()

Async self-play · the clock never stops

ExpertReads the whole manual · 23 pages

/// No access ///ManualNot visible to the Defuser

0:00 / 0:00

Amit Parekh^*, Sabrina McCallum^*, Kareem Al-Hasan^*, Malvina Nikandrou, Alessandro Suglia, Ioannis Konstas

Heriot-Watt University · University of Edinburgh

Read the paper Run your model GitHub

Five frontier models. Zero bombs defused.

No model we test—open or closed—successfully defuses a single bomb. Nine out of ten different human pairs could solve at least one.

02:47

The clock never stops.

Models generate tokens in real time.

Defuse or explode.

No rubric or LLM judge needed because the game provides the ground truth.

✕✕

Defuser

sees the bomb

Expert

holds the manual

The only channel between them

is natural language.

The Defuser can point however models want to.

Our KTANE mod uses normalised (x, y) coordinates and outputs segmentation masks, so the Defuser can use coordinates or set-or-marks.

Coordinates

click {"x":0.51, "y":0.48}

Set-of-marks

press "D"

The bomb defusal manual, spread across its pages

The Expert reasons over the full manual.

All twenty-three pages of rules, wiring diagrams, and symbol tables—in context from the first move.

See the manual

No manual

Collaborating,
or just recalling?

Diagnose real collaboration by taking the manual away to see how strong the parametric knowledge is.

It won’t be saturated and retired.

GPTNT runs on Keep Talking and Nobody Explodes—the same bombs, manual, and ticking clock people play against, and nothing is simplified for the models. It inherits the game’s living modding community, so as models improve we add harder modules—and eventually make them do The Centurion.

MODS

Multimodal & Multilingual

Leaderboard

Randomised manual solutions, pass @1

Click any model for its full results →

#	Model	Interact?How did models interact with the game	Real-time(async)?Async: expert and defuser act on independent live clocks — no shared turns.			Turn-taking(sync)?Sync: expert and defuser alternate in lockstep turns.
#	Model	Interact?How did models interact with the game	Missions?Full multi-module missions defused end-to-end.	Modules?Share of individual bomb modules solved across missions.	Any module?Missions where at least one module was solved before failure.	Missions	Modules	Any module
1	GPT-6 Astra	set-of-marks	0%	24%	60%	0%	50%	100%
2	GPT-5.6 Terra	set-of-marks	0%	18%	40%	0%	24%	70%
3	Inkling-Small	set-of-marks	0%	15%	40%	0%	3%	10%
4	Qwen3.8 Flash Next (180B)	set-of-marks	0%	9%	30%	0%	9%	30%
5	Claude Sonnet 5	set-of-marks	0%	6%	20%	10%	12%	20%
6	Qwen3.8 (27B)	set-of-marks	0%	6%	20%	0%	18%	50%
7	Claude Sonnet 4.6	set-of-marks	0%	6%	20%	0%	12%	30%
8	Qwen3.6 (35B MoE)	set-of-marks	0%	6%	20%	0%	6%	20%
9	GPT-5.2	set-of-marks	0%	6%	20%	0%	3%	10%
10	Gemma 4 (26B MoE)	set-of-marks	0%	6%	20%	0%	3%	10%
11	Mistral Small 4 (119B-A6.5B)	set-of-marks	0%	6%	20%	0%	3%	10%
12	Nemotron 3 Nano Omni (30B-A3B)	set-of-marks	0%	3%	10%	0%	3%	10%
13	Qwen3.5 (27B)	set-of-marks	0%	3%	10%	0%	3%	10%
14	Step 3.7 Flash (198B-A11B)	set-of-marks	0%	0%	0%	0%	6%	20%
15	Gemma 4 (31B)	set-of-marks	0%	0%	0%	0%	3%	10%
16	Qwen3.6 (27B)	set-of-marks	0%	0%	0%	0%	3%	10%
—	Human players	—	25%	60%	93%	—	—	—

Ran GPTNT on your own model?

Submit your run and we'll add it to the board. New models and protocols welcome.

Submit results →

Original-solution leaderboard

News

Latest update · 29 Sep 2026

29 Sep 2026

New model results on randomised manuals

The leaderboard now reports results from 16 models on GPTNT’s randomised manual solutions, including first results for Claude Sonnet 5, GPT-5.6 Terra, GPT-6 Astra, Gemma 4, Inkling-Small, Mistral Small 4, Nemotron 3 Nano Omni, Qwen3.6, Qwen3.8, and Step 3.7 Flash. The earlier original-solution leaderboard remains available as a historical comparison.

02 Sep 2026

GPTNT accepted by TMLR 🎉

We’re delighted to share that GPTNT has been accepted by Transactions on Machine Learning Research (TMLR). 🎊

03 Sep 2026

Randomised manual solutions arrive

GPTNT now uses randomised manual solutions. Each benchmark suite selects a rule seed that changes the solution logic for supported modules, and GPTNT compiles a matching manual for the expert agent. In other words, the bomb’s rules and the expert’s instructions still line up—but they are no longer simply the familiar, original KTANE solutions that a model might have encountered in training. This gives us a more direct protection against memorisation, so we no longer need separate contamination checks. Those checks have now been retired.

Replay

Of actual games played by models we tested

Defuser view

1:15

5:00

Step 6 / 27

Strikes 1 / 3

DefuserExpert

FAQ

What is GPTNT?

GPTNT is a benchmark for real-time collaboration between multimodal models, built on the cooperative game Keep Talking and Nobody Explodes. Two agents must coordinate to defuse procedurally generated bomb puzzles against a live countdown. One agent is the Defuser: it has access to the bomb but not the instructions for defusing it. The other agent is the Expert: it holds the instructions manual but cannot see or manipulate the bomb.

Why is this hard for AI agents?

GPTNT combines conditions that are usually studied in isolation: asymmetric information, real-time asynchronous action, visual grounding, procedural reasoning, and sustained multi-turn communication — all at once. Current models break down on tracking state across turns, acting within the time budget, handling ambiguous descriptions, and recovering from mistakes.

What do you release, and do I need the game?

We release everything needed to run the benchmark: the game mod and microservice framework, the fixed suite of mission configurations, the processed manual, the role-specific system prompts, and the single-agent diagnostic evaluations. You do need your own copy of Keep Talking and Nobody Explodes ($14.99, Windows/macOS/Linux). We do not distribute the game or any of its source, as that would violate its license.

What are the "async" and "sync" modes?

The bomb clock is the whole challenge, and the two modes differ in one thing: whether it keeps running while a model thinks. In async (real-time) mode the clock never stops — every token a model generates burns game time, and both agents act in parallel, so reasoning too long means running out of time mid-move. This is the real game, and the real world: time doesn't wait for your turn. sync (turn-based) mode freezes the clock during generation and advances a fixed step per turn, separating whether a model reasons accurately from whether it reasons efficiently — and lowering the barrier to entry by matching the turn-based loop today's models assume.

AsyncReal timethe clock never stops

0:14Running

0:140:070:00

SyncTurn basedfrozen while thinking

0:14

Turn 1DefuserTurn 2ExpertTurn 3DefuserTurn 4ExpertTurn 5Defuser

0:140:070:00

What's the difference between a strikeout and a timeout?

failed mission ends in one of two terminal states: a strikeout occurs when the Defuser makes three mistakes, leading to three strikes, or a timeout when the timer expires before all modules are solved. These are diagnostically different failures: a model can be careful-but-slow or fast-but-reckless.

How do you separate real collaboration from memorised solutions?

GPTNT now uses randomised manual solutions. Each benchmark suite selects a rule seed that changes the solution logic for supported modules, and GPTNT compiles a matching manual for the Expert agent. The bomb and manual therefore remain aligned, but models cannot rely on the original KTANE solutions they may have encountered in training. This removes the need for separate contamination checks, which have been retired.

Will the benchmark get solved and retired?

GPTNT is designed to stay ahead of improving models. New missions can be procedurally generated and reconfigured by varying the time limit and the count, type, and placement of modules. New puzzle types from the game's active modding community can be added without rebuilding the framework, keeping the task space ahead of any fixed training distribution.

Citation

GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes

Parekh, McCallum, AlHasan, Nikandrou, Suglia, Konstas

Transactions on Machine Learning Research · 2026

TMLR

BibTeX

@article{parekh2026gptnt,
  title = {{{GPTNT}}: {{Benchmarking Real-Time Collaboration Between Multimodal Agents}} on {{Keep Talking And Nobody Explodes}}},
  author = {Amit Parekh and Sabrina McCallum and Kareem AlHasan and Malvina Nikandrou and Alessandro Suglia and Ioannis Konstas},
  journal = {Transactions on Machine Learning Research},
  issn = {2835-8856},
  year = 2026,
  url = {https://openreview.net/forum?id=BXJlDltbtq},
  note = {}
}
@article{parekh2026gptnt,
  title = {{{GPTNT}}: {{Benchmarking Real-Time Collaboration Between Multimodal Agents}} on {{Keep Talking And Nobody Explodes}}},
  author = {Parekh, Amit and McCallum, Sabrina and AlHasan, Kareem and Nikandrou, Malvina and Suglia, Alessandro and Konstas, Ioannis},
  journal = {Transactions on Machine Learning Research},
  issn = {2835-8856},
  year = {2026},
  url = {https://openreview.net/forum?id=BXJlDltbtq},
  note = {}
}

Can two AI agents talk each other through defusing a bomb?

Five frontier models. Zero bombs defused.

The clock never stops.

Defuse or explode.

The Defuser can point however models want to.

The Expert reasons over the full manual.

Collaborating, or just recalling?

It won’t be saturated and retired.

Multimodal & Multilingual

Leaderboard

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

Real-time (async)

Turn-taking (sync)

News

New model results on randomised manuals

GPTNT accepted by TMLR 🎉

Randomised manual solutions arrive

Replay

FAQ

Citation

GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes

Collaborating,
or just recalling?