Agents generate massive amounts of data. Traditional databases weren’t built for this scale, which means teams either wait minutes for queries or can’t query their production data at all.
Brainstore delivers query times under one second, even across terabytes of data.
Topics clusters every production trace to surface what agents are doing. But 100% coverage only works with a small model that clears the quality bar.
So we worked with @baseten to build a benchmark from real Topics traces, iterated against the failure modes that mattered most,
Use the Braintrust Python SDK for tracing and evaling AI across 20+ frameworks and providers.
Auto-instrumentation for OpenAI, Anthropic, LlamaIndex, OpenAI Agents, CrewAI, DSPy, and more. Add the `braintrust` CLI to run evals with custom test cases and scoring functions.
Evals are the foundation of shipping quality agents, but most engineers don't know where to start.
This session from @AICouncilConf covers what evals are, why they matter, and how to build them into your development workflow.
Use the Braintrust Ruby library to access the full Braintrust REST API from any Ruby 3.0+ application.
Includes automatic retries, idiomatic error handling, and simple API key configuration. Install via Bundler to get started.
Early-stage teams building agents know they need evals and observability. But the cost and time commitment often push it down the priority list.
Braintrust for Startups gives early-stage companies access to the same platform that Lovable, Browserbase, and Greptile run in
Use the Braintrust .NET SDK for tracing and evaling AI in C#.
Install the core package plus provider integrations for OpenAI, Anthropic, or the Microsoft Agent Framework. Run evals with custom test cases and scoring functions, or add automatic instrumentation via OpenTelemetry.
Topics has a dedicated page that shows all clusters generated from your production logs.
Compare past and present groupings to understand how user behavior evolves over time and gain visibility into how Topics categorizes conversations, complaints, and feature requests.
I’ve seen a lot of confusion about how to best leverage the new family of GPT 5.6 models, and honestly, sameeee! Running this eval helped me pin down which models I should use (or have an agent use) to build effective code without burning extra tokens and dollars.
We evaled the GPT-5.6 family, plus Anthropic's Fable, Opus 4.8, and Sonnet 5, on the key building blocks of agentic workflows.
Then we broke results down by task type and difficulty and turned them into a decision map you can route against.
Here's what we found.
We evaled the GPT-5.6 family, plus Anthropic's Fable, Opus 4.8, and Sonnet 5, on the key building blocks of agentic workflows.
Then we broke results down by task type and difficulty and turned them into a decision map you can route against.
Here's what we found.
The family-level results show data transforms are nearly solved for the OpenAI models, symbolic rules pull them apart, and the Anthropic rows are dragged down by refusals rather than by wrong answers.
