Pinned
Claude Fable 5 debuts at #1 on DeepSWE.
It outscores the previous best by 3% and sets a new state-of-the-art on our long-horizon coding benchmark.
00:00
Datacurve
19 posts
Datacurve
@datacurve
Research and data to advance frontier models.
- GLM 5.2 is now on DeepSWE as the top open-source model on our leaderboard. With a pass@1 score of 44% at max effort, GLM 5.2 is indisputable #1 open-source model besting Kimi K2.7 Code by 17%.
00:00 - Replying to @datacurveFable 5 scores 70% pass@1 and tracks GPT-5.5 on cost-performance at the default high effort. Kimi K2.7 also joins the leaderboard with a score of 31%.
00:00 - Datacurve reposted
We've updated the Artificial Analysis Coding Agent Index, replacing SWE-Bench Pro with Datacurve's DeepSWE benchmark - the swap lifts Codex with GPT-5.5 (xhigh) above Claude Code with Opus 4.8 (max), while the newly released Claude Fable 5 (max) in Claude Code debuts at the top
- Opus 4.8 is now on DeepSWE. On the default high thinking effort, it scores 6% higher than Opus 4.7 xhigh, while also lowering average cost per task.
00:00Opus 4.8 delivers efficiency gains by solving tasks in fewer steps, directly reducing the total number of input tokens required per task.
Full deep dive coming soon. Check out the full benchmark here → - Datacurve reposted
DeepSWE reflects what I’m hearing from engineers better than any other benchmark. They took the hard path to build a good one.
00:00
Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks. On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work. - Datacurve reposted
Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks. On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work. - Datacurve reposted
Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks. On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work. - Datacurve repostedI presented today at Demo day Day 2 and @TechCrunch featured us @datacurve! Just been reading TC and listening to TC Daily Crunch since high school mornings... a surreal feeling to see us on it. Also, post-demo sadness cuz now YC is coming to an end
