Scale Labs (@ScaleAILabs) / X

Scale Labs

144 posts

Scale Labs

@ScaleAILabs

welcome to the lab. from the researchers at @scale_AI

Joined October 2025

Scale Labs reposted
Mohit Raghavendra (@ ACL)
@mohit_r9a
Jul 10
Excited to see our SWE Atlas - Codebase QnA benchmark featured in Meta's release for Muse Spark 1.1. This benchmark was specifically designed to test deep codebase understanding and agentic runtime analysis -- quite different from other SWE benchmarks, and more aligned with
Alexandr Wang
@alexandr_wang
Jul 9
1/ muse spark 1.1 is an industry-competitive agentic and coding model. across many agentic evals it rivals gpt-5.5 and opus-4.8. available now through the new meta model api and in meta ai. 🧵
Scale Labs
@ScaleAILabs
Jul 10
We appreciate the community's feedback on SWE-Bench Pro. Much of it maps to changes already underway in v1.1, which we've been building for a while. Keeping evals current with frontier models is hard, and we're always iterating. SWE-Bench Pro Verified coming soon. 👀
Scale Labs reposted
Chetan Rane
@the_chet2an
Jul 4
At #icml2026 in Seoul next week! We’ll be covering 7 accepted papers, including SWE-Bench Pro. Come find me to chat about RL envs, computer use, auto research, AI for expert work in finance, science, healthcare. Folks at @scale_AI are also hosting a cocktail reception,
Scale Labs reposted
Sri Kundurthy
@srkvatsa
Jul 3
I’ll be presenting our work on on-policy expert corrections (OEC) at ICML next week! grateful for the team led by @NiklasLauffer, with Xiang Deng, @Bckenstler and @_jeffda
Scale Labs
@ScaleAILabs
Jul 2
Replying to @ScaleAILabs
Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections arxiv.org/abs/2512.14895
Scale Labs
@ScaleAILabs
Jul 2
ICML 2026 is almost here, and we're headed to Seoul with 7 accepted papers. 🇰🇷 Our research spans agent training, scientific reasoning, and evaluations advancing the next generation of AI systems. Here's an early look at the work we'll be presenting 🧵
Scale Labs
@ScaleAILabs
Jul 2
Replying to @ScaleAILabs
RubricRobustness: Evaluating the Sensitivity of Rubrics-Based Benchmarks to Simple Perturbations openreview.net/forum?id=2Y3I3…
Scale Labs
@ScaleAILabs
Jul 2
Connect with us at ICML: scale.com/events/icml-20…
Scale Labs
@ScaleAILabs
Jul 1
Less than a year ago, we launched the Remote Labor Index with @CAIS to measure how well AI agents can complete paid freelance work. Since then, the best automation rate has jumped from less than 3% to 16%. We've also added results for three frontier models to the leaderboard:
00:00
Scale Labs
@ScaleAILabs
Jul 1
Full leaderboard:
Remote Labor Index (RLI)
From labs.scale.com
Scale Labs
@ScaleAILabs
Jun 30
AI is transforming drug discovery, but until now there hasn't been an independent benchmark for the computational tasks behind early-stage research. Together with @phylo_bio, we built DrugDiscoveryBench to evaluate how today's leading AI models perform on the core tasks
00:00
Scale Labs
@ScaleAILabs
Jun 30
More on what we learned: scale.com/blog/drugdisco…
Scale Labs reposted
Mohit Raghavendra (@ ACL)
@mohit_r9a
Jun 30
📝 New research from @scale_AI Frontier SWE benchmarks are usually single-turn, one-shot tasks: the agent gets a detailed spec upfront, then implements autonomously. That is not how most real coding-agent workflows feel. Introducing SWE-Interact. 🧵
Scale Labs
@ScaleAILabs
Jun 24
GLM 5.2 is now live across all three SWE Atlas leaderboards: Codebase QnA, Test Writing, and Refactoring. When we launched SWE Atlas a few months ago, open models were significantly behind frontier closed models, clustering at the bottom. GLM 5.2 has leapfrogged this gap, and is
Scale Labs
@ScaleAILabs
Jun 24
More results across SWE Atlas leaderboards:
AI Model Leaderboards & Benchmarks
From labs.scale.com