Introducing the Firecrawl Developer Index, built for supercharging coding agents. Read the announcement →

Firecrawl Benchmarks: open retrieval evaluations

Benchmarks
Overall recall · higher is better · scale 0 to 100%
ImageFirecrawl Developer Index63.1%
ImageParallel57.7%
ImageMintlify54.6%
ImageExa53.7%
ImageNative web search45.4%
+3 more in the full table

Developer Retrieval Benchmark

An agent runs the same developer questions through each system with one search tool and ten results per call, then scores whether the right answer is cited across three tracks: the repository behind a described capability, the pull request that fixed a bug, and the docs page that answers a how-to. Compare Recall@10 per track and MRR@10 overall.
[ 8 systems ][ 1,179 tasks ]
Aug 21, 2026
Also published

Firecrawl runs these too. They have no page of their own yet: the numbers and the method are published on the pages that cite them.

Scrape coverage and quality, run Jan 13, 2026

1,000 URLs from ten categories of public web pages, scored on whether each tool returned the core page text.

Coverage (success rate)
96%
Extraction accuracy (F1)
0.638
Content recall
0.639
Latency (P95)
3,387 ms

Dataset and scoring

1,000 URLs drawn from diverse public web domains (news, documentation, e-commerce, finance, and more). A URL counts as covered when the tool retrieved at least 10% of the expected content, defined as core page text excluding navigation, ads, and footers.

firecrawl/scrape-content-dataset-v1

Third-party studies

Benchmarks Firecrawl appears in but did not run. The figures, the method and the scoring belong to whoever published them.

Openbenchmarks coding tickets, Sep 1, 2026 snapshot

An independent search-only comparison of 11 web search APIs on 100 held-out documentation tickets. Firecrawl places second behind Perplexity.

Grounded task completion
70.3%

Scope

Search-only: every vendor returns snippets and nothing opens a page. The study publishes a separate search and fetch board on which the ranking differs.

Openbenchmarks web search study · openbenchmarks-labs/web-search-for-coding-agents

Openbenchmarks multi-hop discovery, Sep 1, 2026 snapshot

The agent half of the same study: 45 multi-constraint discovery questions, search-only. Firecrawl places twelfth of sixteen configurations, well behind Parallel.

Multi-hop F1
30.4%

Scope

Search-only, with no page opened, on questions that need several searches to assemble a complete set. It is a different task set from the study's coding tickets, so the two rows are not comparable to each other.

Openbenchmarks web search study · openbenchmarks-labs/multi-turn-company-search

How we benchmark

Same input for everyone

Every system gets the same inputs in the same order with default settings. No per-system tuning and no dropping the cases a system struggles with.

Scoring you can check

Metrics are computed against human-annotated answers, not judged by a model, so the same run produces the same score. Every metric a benchmark page carries is defined in full on that page.

Our own numbers included

Firecrawl runs these benchmarks and appears in them. Every result is published, dated, and changelogged so the comparison can be argued with rather than taken on faith.