EMNLP 2026 Tutorial October 2026

Failure Modes in Agentic AI

From Reproducible Triggers and Diagnostics to Reliable Fixes

Zoey Sha Li, Zihan Wang, Hejie Cui, Cheng Qian, and Manling Li

Abstract

Foundation-model agents are being deployed faster than the community can characterize how they fail. In barely two years they have moved into browsing, software engineering, scientific discovery, and embodied control, where reliability is governed by long-horizon, closed-loop interaction that single-turn evaluation rarely captures. Failures already reaching users (error cascades, reward hacking, mode and reasoning collapse, memory drift, and over-reasoning loops) still lack a shared vocabulary, reproducible triggers, and comparable diagnostics.

This tutorial gives a systematic, mechanism-first account of agentic failure modes across four axes: environments and infrastructure, training dynamics, test-time scaling and adaptation, and evaluation and red teaming, with safety threaded throughout. Attendees leave with reproducible triggers, comparable diagnostic protocols, and verifiable mitigations.

Schedule

Three and a half hours, including a 30-minute break. Times are counted from the start of the session.

Time	Session	Presenter
0:00 - 0:10	Part 1 The Development Loop of AI Agents What an agent is and how one is built: the closed agent loop, the current agent landscape, and a build lifecycle that maps onto Parts 2 to 5. Cheng Qian and Zoey Sha Li	Cheng Qian and Zoey Sha Li
0:10 - 0:50	Part 2 Agent Environments and Infrastructure The tool-call protocol, getting a model to call tools, and the RL system and environments behind agent training; then three failures at the seams between model and harness: token masking, re-tokenization drift, and environment errors scored as the policy's. Zoey Sha Li	Zoey Sha Li
0:50 - 1:30	Part 3 Training Dynamics Collapse, reward hacking, and the missing success signal: how the learning signal fails, how to see it, and how to restore it. Safety: agents that game their benchmarks. Zihan Wang	Zihan Wang
1:30 - 2:00	Coffee break
2:00 - 2:40	Part 4 Test-Time Scaling, Adaptation, and Recursive Self-Improvement When more reasoning lowers accuracy, how memory goes stale or collapses when compressed, how a self-improving agent anchors on its past and misses its own regressions, and autoresearch agents that hack their own evaluations. Hejie Cui	Hejie Cui
2:40 - 3:20	Part 5 Evaluation, Safety, and Open Frontiers Where static benchmarks break, closed-loop and contamination-resistant evaluation, red teaming of agent attack surfaces, and open frontiers. Manling Li	Manling Li
3:20 - 3:30	Synthesis and Q&A Which fixes transfer, which are domain-specific, and what diagnostic infrastructure the community needs most; release of reproductions and trace examples. Manling Li	Manling Li

Tutorial Outline

Parts 2 to 5 each follow the same arc: failure patterns, diagnostics, fixes, and open challenges.

Part 1 10 min Zoey Sha Li

The Development Loop of AI Agents

An agent is not a single LLM call but a closed loop: it perceives, plans, acts through tools, and observes over many steps while persisting state in memory. We survey the landscape and lay out a build lifecycle that maps onto the rest of the tutorial.

Agent landscape

Open-source personal assistants (OpenClaw, Hermes Agent)
Coding agents (Devin, Claude Code, Codex)
Browser and computer-use agents (ChatGPT Work, Claude Computer Use)
Deep-research agents (OpenAI deep research, Perplexity Deep Research)
Embodied agents (π0.7)

Build lifecycle

Scaffold and tools for multi-turn trajectories (Part 2)
Post-training with SFT and RL (Part 3)
Memory, skills, and test-time adaptation (Part 4)
Evaluation, safety, and open frontiers (Part 5)

Part 2 40 min Zoey Sha Li

Agent Environments and Infrastructure

Training an agent means turning what the model writes into calls, tokens, and rewards. We cover the tool-call protocol, how SFT and rewards get a model to call tools at all, and the RL system and environments behind agent training: search APIs, code sandboxes, and a container per software task. Every failure in this part sits at a seam between the model and its harness.

Failure patterns

Cold start: the model barely calls its tools, so on-policy RL has little to reinforce
Token misattribution: the environment's tokens end up in the loss
Re-tokenization drift: decoding, parsing, and re-encoding a turn changes the token IDs the trainer sees
Environment errors scored as the policy's: failed environments, empty results that lead to void turns, and slow tools whose timeouts can pass for a change in the policy

Diagnostics

Tool calls per trajectory
Decode the trained tokens and count them against what the inference engine generated
Per-call token counts, and the token IDs when you keep them
Per-tool failure, timeout, and call rates; per-turn checks for void turns

Fixes

An SFT prior whose data covers the behavior (SynPlanResearch-R1), or a reward that pays for each part of a call (ToolRL)
Build the loss mask with the sequence, 1 only for what the model wrote (Search-R1)
Token in, token out: ask the engine for token IDs and tokenize each new message on its own
Rerun or drop trajectories the environment broke, filter trajectories with void turns (SimpleTIR), and keep timeouts out of the reward

Open challenges

Environments that are expensive to run: computer use, scientific experiments, and robotics
Building environments at scale, and checking the verifiers that come with them (SWE-Universe)

Part 3 40 min Zihan Wang

Training Dynamics

Three failures of the learning signal: collapse, where useful variation disappears; reward hacking, where optimization drifts from the intended objective; and no success signal, where no successful rollout enters training.

Failure patterns

Collapse: training gains disappear, success can reinforce a false explanation (RAGEN), and high entropy can hide generic, input-agnostic reasoning (RAGEN-2)
Reward hacking: more proxy reward can lower true quality (Constrained RLHF); agents retrieve answers from git history, move work outside a timed check, and pass hacks to other agents
No success signal: when every rollout in a group fails, centered rewards are zero and nothing is reinforced

Diagnostics

Input dependence: can a reasoning trace identify its own input? (cross-input scoring, RAGEN-2)
Outcome contrast: within-group reward variance, and noise from re-tokenization and training-inference mismatch
Proxy points found with independent evaluation; re-running fresh agents after sealing the answer channel
Reward histograms that tell all-fail groups from all-succeed ones

Fixes

Filtering for informative prompt groups (RAGEN-2), exploration control at the update, token, and turn level (DAPO, T²PO), and importance-sampling correction for engine mismatch
Component targets for composite rewards (Constrained RLHF); measure the intended operation and repair the grader that actually runs
Partial and denser rewards (VAGEN, BAGEN), more samples on unresolved prompts (NGU), external information (LUFFY, POPE, NuRL), and dense-to-binary reward curricula

Open challenges

Useful learning signal without narrowing task coverage
Separating environment randomness from learning signal
Testing whether reasoning is causally faithful to the input

Part 4 40 min Hejie Cui

Test-Time Scaling, Adaptation, and Recursive Self-Improvement

Four places where deployment-time improvement fails: test-time scaling, where more reasoning can lower accuracy; memory and skills, where what the agent keeps goes stale or makes it worse; recursive self-improvement, where the agent edits its own harness; and autoresearch, the same loop pointed at research.

Failure patterns

More reasoning can lower accuracy, and agents misjudge how much budget they have left (BAGEN)
Memories go stale without being contradicted (STALE), and one rewrite that compresses a long context can leave the agent worse than no context at all (ACE)
Long search histories anchor a self-improving agent to its past attempts
Autoresearch runs leave flaws they notice unfixed, overclaim, and hack their own evaluations

Diagnostics

Accuracy against tokens, turns, tool calls, and improvement rounds
Probes of compressed memory beyond the compression ratio (context collapse, ACE)
Hidden-set replay of the final artifact (RSI-Exam), and regression checks on tasks the improver did not predict (AHE)

Fixes

Budget-aware stopping and budget trackers (BAGEN, Budget Tracker), and length penalties on steps that do not move the answer (SWAP)
RL-trained memory managers (Memory-R1, MemAgent), context folding, and strategies distilled from successes and failures (ReasoningBank)
Small edits kept only on held-out improvement, evolved on benchmark-disjoint tasks (ModularRSI, SkillOpt)
Keep the whole research record: agent-native research artifacts with the dead ends and the evidence behind each claim

Open challenges

Attribution: which behavior did each harness edit actually change?
Predicting what an edit will break as well as what it fixes
Edits that compose without interfering
What a search loop should remember: shared memory helps, long history anchors
Oversight that holds when research agents learn from their reviewers
Verification beyond the benchmark the loop was tuned on

Part 5 40 min Manling Li

Evaluation, Safety, and Open Frontiers

Leaderboard scores routinely fail to measure what we think. We show where static benchmarks break, how closed-loop evaluation restores signal, and how red teaming probes agent attack surfaces, including those flagged in Parts 2 to 4. The part closes with open frontiers and the synthesis.

Failure patterns

Contamination: test instances leak into training, so the score reflects recall
Saturation: a benchmark stops discriminating as models reach its ceiling (SWE-bench)
Construct invalidity: the benchmark does not isolate the intended ability
Harness variance: the same backbone scores differently across scaffolds and tools
Attack surfaces: prompt injection, reward hacking under safety constraints, memory poisoning

Diagnostics

Black-box tests for test-set contamination
Harness-controlled comparisons of one backbone across scaffolds (Harness-Bench)
The attack surface as a measurable axis under adversarial inputs (Agent Security Bench)

Fixes

Dynamic, live, and asynchronous evaluation that resists memorization and restores headroom (ARE/Gaia2, Online-Mind2Web)
Social, embodied, and out-of-sync recovery evaluation (SOTOPIA, SyncMind)
Verifiable safety-policy reasoning as a runtime defense (ShieldAgent)