ReactBench

ReactBench is an evaluation for coding agents on realistic React work. Today’s models can pass tests in benchmarks but still write React that fails in production. Tests verify behavior, but they miss React performance, accessibility, and quality issues.

Here’s what ReactBench does differently:

• A higher bar than just passing tests

  Every solution must pass both behavioral tests and React Doctor, our open-source, deterministic verifier for React code. Its 400+ rules scan the agent's code for broken effects, unnecessary renders, accessibility problems, and maintainability issues.

• Built by React experts

  Our team built React Doctor ★ 13.8k, React Scan ★ 21.6k, and Million.js ★ 17.7k, used by engineers at GitHub, PayPal, Rippling, and Airbnb. We spent hundreds of hours curating tasks to measure real gaps in model capabilities.

• Real work in real codebases

  ReactBench spans open-source repositories and realistic changes grounded in existing projects (not synthetic puzzles). Agents must implement, preserve behavior, and meet the React-specific quality checks.

Results

We evaluate two complementary capabilities: implementing new features and improving existing code.

• Writing React

  Implement a real feature or fix, pass held-out behavioral tests, and validate no new React Doctor issues were introduced. See example

• Fixing React

  Refactor React failures from existing code while preserving existing behavior without introducing new React Doctor issues. See example

We report pass@1 averaged across 5 trials per task. Error bars are 95% run-to-run intervals: each trial index forms one complete pass over the fixed task set.

ReactBench remains unsaturated. GPT 5.6 Sol at Max has the highest observed aggregate score at 47.8%. The next four configurations are GPT 5.6 Sol at XHigh at 43.9%, Opus 5 at Max at 43.5%, Fable 5 at XHigh at 43.1%, and Fable 5 at Max at 42.0%.

We also evaluated task discrimination across model configurations. In the 51-task panel, 39 tasks (76.5%) had a cross-configuration standard deviation above 0.10.

Among configurations within 5.0 percentage points of the leader, GPT 5.6 Sol at XHigh has the lowest average cost. It reaches 43.9%, 3.9 percentage points behind the leader, while costing 36.0% less per trial ($2.32 versus $3.62).

GPT 5.6 Sol at Max achieved the highest observed pass rate. Its 95% run-to-run interval overlaps GPT 5.6 Sol at XHigh’s, so this revision does not identify a clear winner with confidence.

Across 8,640 Writing React trials, the evaluated models introduced 7,409 graded React Doctor issues. Bugs, including security findings, accounted for 6,682, or 90.2%.

Harbor exposes component verifier rewards for 4,826 of 5,558 failed Write trials and 5,343 of 6,116 failed Fix trials in ReactBench v1.1. Among the covered Write failures, 2,279 failed behavioral tests only, 1,303 failed React Doctor only, and 1,244 failed both gates. Among covered Fix failures, the corresponding counts are 1,127, 3,035, and 1,181.

The remaining Harbor jobs do not expose gate-level rewards publicly, so these partial splits are not presented as estimates for the full revision.

Why we built ReactBench

React is the dominant frontend framework and the most popular target for coding agents. Roughly 70% of websites built on a JavaScript framework choose React.

We have seen the risks firsthand. React Doctor is our open source tool for scanning React issues used by engineers at PayPal, Rippling, Polymarket, and the Centers for Disease Control and Prevention (CDC). Adoption is largely driven by the increase of model-generated code that makes it easier for subtle defects to reach production. As models write more React, small mistakes can propagate at enormous scale. In the worst cases, these defects lead to production failures:

• Outages

  Incorrect useEffect usage takes down production. Cloudflare traced its September 2025 dashboard and API outage to one effect with a faulty dependency. Despite human review and test, the bug still shipped to production.

• Lost revenue

  Slow interfaces cost sales. Google and Deloitte found that a 0.1s mobile speedup increased retail conversions by 8.4%. Rakuten 24 increased revenue per visitor by 53.4% after improving Core Web Vitals.

• Legal risk

  Interfaces that are not accessible exclude customers and expose companies to lawsuits. WebAIM found automated accessibility failures on 95.9% of the top one million home pages and US federal web accessibility lawsuits increased by 27% in 2025.

Building ReactBench

Each task type uses a different construction and evaluation process.

Writing React

Writing React measures whether an agent can implement real work without introducing a React regression. Each task starts from a merged pull request in an open-source repository. The agent receives the base repository and an issue-style instruction. The reference patch and verifier are not provided.

• Behavior

  The verifier injects the hidden behavior tests and verifies the results in a separate container after the agent completes.

• React health

  A pinned React Doctor scan compares the submission with the base commit. If a new issue is detected, it fails.

Fixing React

Fixing React measures whether an agent can recognize and refactor React problems from source code alone. We select a component with known React issues and ask the agent to improve it without naming those findings.

• React health

  The agent must remove every target without introducing another graded React issue. The grader ignores line shifts.

• Behavior

  The task’s test suite must still pass to ensure refactoring the component did not regress it in functionality.

We strip React Doctor and other React-aware linters from the agent image. Only the verifier contains the pinned scanner and its clean baseline.

React Doctor as a verifier

We built React Doctor, an open-source, deterministic React verifier with 400+ rules to cover problems outside behavioral test assertions:

• Correctness

  Catches conditional hooks, unstable list keys, hydration mismatches, and deprecated React APIs.

• State and effects

  Flags derived or duplicate state, useEffect abuse, and infinite rerenders.

• Performance

  Finds unnecessary renders, layout thrashing, sequential async work, and bundle-heavy imports.

• Accessibility and security

  Detects unlabeled controls, keyboard-inaccessible interactions, unsafe HTML, and secrets exposed to client bundles.

ReactBench does not use every rule or an LLM-as-a-judge. We curate the specific React Doctor rules used for grading, then pin the scanner version and clean baseline.

From real pull requests to verified tasks

ReactBench converts merged changes from public repositories into evaluation tasks.

• Mine candidates

  We collect merged pull requests from open-source React projects.

• Filter candidates

  We built automated filters to check if there were meaningful changes to product code. Then, reviewers assess exceptions that cover broader feature work or important projects, and have realistic behavioral tests.

• Author tasks

  Each reviewer converts validated pull requests into a realistic issue-style instruction. If the candidate’s tests are too strict, reviewers will rebuild a custom test harness to validate behavior.

• Validate behavior

  The tests must fail against the pinned base commit and pass with the reference solution. The unchanged repository must score 0, while the reference solution must score 1.

• Test the verifier

  An adversarial agent attempts to earn full credit without implementing the requested behavior.

• Pin the release

  For each task, we pin the source commit, environment, test suite, reference solution, and verifier configuration. We also have a versioned manifest that records a hash for each published artifact.

Filter criteria

92% percent of Writing React tasks meet both the recency and repository-size criteria. These thresholds are implemented to reduce likely training exposure.

Behavioral coverage

Every task defines deterministic behavioral checks before release. We use the repository’s existing Vitest, Jest, or Playwright tests when they fully specify the requested behavior.

When existing tests leave gaps, we build a separate harness around observable behavior. Reviewers confirm that the verifier accepts valid implementations beyond the reference solution.

Clean-room grading

The agent and verifier run in separate containers. Before grading, the verifier restores its own Git metadata, dependencies, protected configuration, hidden tests, and pinned React Doctor binary.

Continuous integration (CI) confirms that both containers use the same source commit. The verifier runs offline and doesn’t make external network requests. The agent cannot access hidden tests or the reference solution during its run.

Separating model failures from benchmark failures

A zero can reflect a model failure, unclear instructions, a brittle test, a verifier error, an infrastructure failure, or an invalid run.

Reviewers inspect the trajectory and final patch before including a result in model-performance conclusions. They classify each reviewed rollout as a legitimate solve, genuine model failure, verifier false positive, verifier false negative, or invalid run.

Testing ReactBench against reward hacking

The adversarial agent probes the test infrastructure, reward files, Git history, and React Doctor inputs. Its goal is to earn full credit without implementing the requested behavior.

We fix or remove any task that exposes a cheat, then rerun every control. This final check tests whether ReactBench measures the requested work rather than an agent’s ability to exploit the grader.

Limitations

React Bench evaluates agents, not models in isolation. Differences among Codex CLI, Claude Code, Cursor, Gemini CLI, and other harnesses can affect results.

Every task combines behavioral tests with React Doctor as a verifier. These checks catch important React problems, but they cannot guarantee visual correctness and other important attributes.

Also, the benchmark primarily covers open-source React projects. Results may not generalize to proprietary codebases, different architectures, or other frontend frameworks or setups.

React Bench will publish versioned tasks, solutions, manifests, and release records so others can inspect and reproduce each evaluated task set.

Future work

We plan to compare more models, on more effort levels. Additionally, we plan to run not only on the commercial harnesses, but also on mini-swe-agent to get a more solid baseline.

We also plan to expand into visual design and other frontend frameworks and diversify the repositories and task types represented.

We’re constantly improving ReactBench. Report benchmark issues on GitHub. Model labs and coding-agent teams can reach out to evaluate models on our held-out task set, get early access to new benchmarks, or collaborate with us.

Acknowledgments

ReactBench exists because engineers and open-source maintainers contributed their time and expertise. We thank everyone who reviewed its tasks, drafts, and evaluation infrastructure.

• Research

  Aiden Bai, Nisarg Patel, Rasmus Gustafsson, Ray Arayilakath

• Design

  Ben MacLaurin

• External contributors

  • Michał Pierzchała(React Native team)
  • Jovi De Croock(Preact core team)
  • Dev Agrawal(SolidJS core team)
  • ryoppippi(creator of ccusage)
  • Rahim Alwer(creator of Vidstack, Video.js v10 team)
  • Tiger Abrodi(early Lovable team)
  • Isabelle Reksopuro

• Data Advisory

  Parth Patel (Founding Engineer at AfterQuery)

Special thanks to the engineers who built, verified, and reviewed ReactBench’s tasks and evaluation infrastructure, and to the maintainers of the open-source repositories represented in the current task set.