Today we're releasing HiL-Dynamics, the first open-source tool that measures how production agents actually collaborate with humans under uncertainty. Not just whether they got the answer.
Now you can measure exactly when your agent asks for help, when it makes assumptions, and
Scale AI
2,263 posts
- To understand our story, you have to go back to the beginning. It started with self-driving cars. Ten years later, it's the architecture underneath AI that actually works, across frontier labs, enterprises, governments, and mission-critical systems around the world.
00:00 - Scale AI repostedThe humans stay. That’s the idea behind @scale_AI's new brand campaign. 10 years of building AI has taught us something: the most important decisions belong to humans. The AI that works in decisions of consequence keeps humans at the center. Going live in SF and NYC. Where to
- This month we turn 10. The hard work started in 2016, and it hasn’t stopped. Shortcuts are for losers. Winners welcome. scale.com/careers
00:00 - Scale AI repostedToday we’re releasing Refactoring, the final leaderboard of our SWE Atlas suite. This new leaderboard is the ultimate test of an agent's ability to restructure code without breaking the system. Claude Opus 4.7 with Claude Code takes the top spot🥇
- Proud to share @CDAODoW has expanded its enterprise agreement with Scale AI raising the ceiling from $100M to $500M. This expansion reflects our continued commitment to accelerating the adoption of AI capabilities across the Pentagon to help America stay prepared, resilient,
- Scale AI repostedAI pretenders vs. AI contenders. It's those who still haven’t realized reliability is the product vs. those who can deliver reliability and outcomes. That's what the enterprise AI race comes down to. Here's a note I sent the Scale team this week.
- Scale AI repostedWe recently built HiL-Bench, the first benchmark to test a critical question: do AI agents know what they’re missing and when to ask? Frontier models perform well with perfect specs. But remove a few key details, and they confidently guess and ship plausible wrong answers. We
- Scale AI has acquired ICG Solutions, a defense technology firm specializing in real-time streaming data analytics. This is another step forward in how we support the U.S. defense and intelligence community with AI systems built to serve America’s most important national security
- Replying to @scale_AIPaper: static.scale.com/uploads/67a153… Data: huggingface.co/datasets/Scale… Leaderboard: labs.scale.com/leaderboard/hil Code & Harness: github.com/hilbenchauthor…









