Braintrust

Star and tag datasets for easier discoverability and organization across your team.

Want to run an eval on a multi-turn conversation between a person and AI? Instead of scoring each span independently, Braintrust can score an entire trace, so you can measure the quality of a complete interaction.

Reliable scorers are how evals help build quality AI products. Create single-aspect scorers for each evaluation dimension, provide clear rationale for LLM-based judgments, and use weighted scoring to prioritize critical criteria.

Systematically improve your prompts with bulk annotation in Braintrust. Rank multiple chatbot responses with notes on why they're good or bad, ask an LLM to analyze your annotated comments and suggest prompt improvements, then make changes to your prompts based on those suggestions.

Parameters now support enums for cleaner, more reliable eval configurations. Define dropdown options in Python or TypeScript, and Braintrust automatically generates UI controls with validation.

Braintrust is heading to London. Join us at AI Engineer Europe April 8-10 for keynotes, workshops, and 1-1 meetings. Then enjoy an AI night at the London aquarium with ElevenLabs, Modal, and turbopuffer. DJ, food, drinks, and 5K+ underwater friends. More here → https://lnkd.in/gkB2y-gM

Congratulations to Basis, Browserbase, Granola, Lovable, Modal, Vercel, and all the friends and customers on this year's Wing Venture Capital Enterprise Tech 30. See the full list → wing.vc/et30

Thank you Ramp.

The trace tree now shows estimated LLM costs inline on each span, with costs automatically propagated from child to parent spans. See which parts of your multi-step workflows consume the most budget at a glance.

For an AI product, the most important question is simple: is it getting better? That's why evals are the new PRD. They specify desired behavior, define acceptance criteria, track progress, and prevent regression in a form that works for non-deterministic systems.