Arize AI

@arizeai

The AI engineering platform for teams shipping reliable AI agents and LLM applications. Also home to @ArizePhoenix.

San Francisco, CA

arize.com

Replies Reposts Media Articles

Pinned
Arize AI
@arizeai
Sep 26
Jev-as-a-Judge is live in Arize AX. Ask most LLM judges a yes/no question and you get a paragraph back. Then you parse it to figure out the answer. Jev returns typed outputs instead: booleans, choices, or rubric scores, with confidence on classifications. You can also evaluate
5
Arize AI
@arizeai
19h
ICYMI: Moving past the hype with TypeSafe AI's Jev model, how we used a managed agent to find 43 unnecessary tool calls, a conversation with Daytona's founder Ivan Burazin, and what's new in Arize AX.
Article
What Jev is teaching us about better evals
We've been busy testing TypeSafe’s Jev across tens of thousands of evaluations. One result stood out: Jev’s confidence was a much stronger signal of its own mistakes than repeatedly running an LLM...
7
Arize AI
@arizeai
Sep 28
One Jev call gave us a better error signal than rerunning an LLM judge 10 times. Jev’s probability reached 0.95 ROC AUC for identifying incorrect judgments. Disagreement across 10 repeated LLM judge calls reached 0.62-0.84. We tested this across 36,190 judgments and 10 Phoenix
1
Arize AI
@arizeai
Sep 24
When using an LLM-as-a-judge, or playing with prompts, you need access to the models you want at high speed, including open source, and your fine-tuned models. Arize AX now has support for @FireworksAI_HQ as an AI provider, so you can use all your favorite models.
Fireworks AI - Arize AX Docs
From arize.com
1
Arize AI
@arizeai
Sep 24
Every session with an AI assistant starts from zero. You have to re-explain the goal, the error column, and the spike that turned out to be test traffic. Alyx now remembers your goals, the signals you watch, what you already ruled out, and more. Long-term memory docs -->
Alyx: Arize AX's AI engineering agent - Arize AX Docs
From arize.com
1