Vals AI

@ValsAI

Public LLM Evaluation // vals.ai @8vc @BloombergBeta @pearvc @a16z @HRTVentures @nextladder

San Francisco, CA

vals.ai

Replies Reposts Media

Pinned
Vals AI
@ValsAI
Sep 1
Using Fable 5.1, we’ve elicited a decoding of a cipher that has been unsolved for 373 year, and listed as #28 on Klaus Schmeh’s (the most-published cryptology author in the world) “The Top 50 unsolved encrypted messages.” The solution? Almost embarrassingly simple in hindsight.
Made with AI
44
Vals AI
@ValsAI
8h
Every major industry has an independent third evaluator and this is where AI is lacking. @RayanKrishnan makes the case on @a16z with @bhorowitz and @JenniferHli.
a16z
@a16z
10h
Vals AI co-founder and CEO Rayan Krishnan with a16z's Ben Horowitz and Jennifer Li on grading AI, what it costs, and who gets to make the rules: Every big industry eventually grows an independent testing layer. AI has credit ratings to learn from and Enron to avoid. Model
00:00
Vals AI
@ValsAI
Sep 8
SUPERHOT is another game in our public set of games. Astra plays surprisingly well. At around 2 hours and 7 minutes, it punches an NPC, steals their gun in mid air, then shoots them.
00:00
Vals AI
@ValsAI
Sep 8
03:01:05
Astra began to saturate a computer-use eval we haven’t even released yet. Astra was not only able to kill several hostile mobs in Minecraft, but it made it all the way to building a Nether portal in less than three hours. We believe that if we ran it longer, it would beat the
1
Vals AI
@ValsAI
Sep 8
Astra began to saturate a computer-use eval we haven’t even released yet. Astra was not only able to kill several hostile mobs in Minecraft, but it made it all the way to building a Nether portal in less than three hours. We believe that if we ran it longer, it would beat the
00:00
13
Vals AI
@ValsAI
Sep 8
GPT-6 Astra lands at #3 on the Vals RSI Index, ahead of GPT-5.6 Sol but behind Fable 5.1 and Opus 5.
GIF
2