Irregular

Anthropic used CyScenarioBench, our benchmark for multi-stage offensive cyber operations, in the cyber evaluations for Claude Opus 5.5. Across a ten-challenge subset, Opus 5.5 averaged a 67.6% solve rate, ahead of Claude Mythos 5.1 at 61.7% and Claude Opus 5 at 53.0%. Full Show more

As open-weights models become more capable, we expect more organizations to run them in-house, using the same model to power both applications and the coding agents that maintain them. In our latest research, we observed a phenomenon we call agentic self-modification: an agent Show more

Introducing SOLVE+: Irregular’s scoring system for the difficulty of full offensive cyber scenarios. For the past 18 months, frontier AI labs have used SOLVE to score vulnerability research & exploit challenges. SOLVE+ expands that scope to reconnaissance, operational security, Show more