There's a lot of talk about loops recently.
But the term “loop” currently describes at least four different architectures: execution, task, product, and system (plus the human oversight loop governing them).
Execution loops are the loop most people picture when they say "agent."
But there's more to this space than just that.
- Execution: steps in one run
- Task: fresh runs against a spec
- Product: agents across repo/backlog
- System: improve prompts/evals/harnesses
Your eval suite is incomplete right now. Guaranteed.
You wrote it before a single real user touched the agent, so it can't cover the questions they'll actually ask.
GPT-5.6 support just went live in Arize AX. 🚀
Now available:
🌞 gpt-5.6-sol
🌍 gpt-5.6-terra
🌙 gpt-5.6-luna
Compare all three side-by-side in the Prompt Playground, plug them into LLM-as-a-judge evals, and watch them in production - all in one place.
Try it 👇
Before you rip out Kubernetes for something faster, do one thing: trace what you already have.
Half the time the bottleneck isn't your runtime. It's model latency, tool selection, or a retry loop hiding in plain sight.
I found @grinich talk at Observe fascinating because he articulated so well what's fundamentally different about authentication and authorization in the age of agents.
If you are interested in agent first experiences, I can't think of a more dialed in tech leader.
An agent was told: “make the tests pass.”
It deleted the tests.
That story is funny on its face. But it's also the exact reason agent engineering is getting harder.
In this Rise of the AI Engineer conversation, @WorkOS founder @grinich makes the case that the next layer of AI
An agent was told: “make the tests pass.”
It deleted the tests.
That story is funny on its face. But it's also the exact reason agent engineering is getting harder.
In this Rise of the AI Engineer conversation, @WorkOS founder @grinich makes the case that the next layer of AI
Most teams hear the same advice: “add evals.”
But when you’re staring at a real LLM app, that advice gets vague fast.
Should your first eval be an integration test? A golden dataset? A CI gate? A dashboard metric? An LLM judge?
Our answer? Write your first eval like a test.
In a practical writeup, Arize's Head of Open Source @mikeldking walks through exactly how to run LLM evals directly inside pytest, Vitest, or Jest with Phoenix.
Here's what he covers:
- how evals differ from ordinary tests
- what
Pro tip: not every check should break the build.
Hard invariants belong in CI. Quality signals like helpfulness, latency, and groundedness should be recorded, trended, and inspected with traces.
That gives you a practical first eval without turning normal model variance into
Agent harnesses are becoming the durable layer of AI coding workflows, according to @aparnadhinak.
The model answers once. The harness turns that answer into a loop: context, tools, permissions, edits, tests, failures, retries, recovery, and traces.
That loop decides how
The difference between an agent that works and one that games you comes down to one habit: a good eval.
✅ Spell out the shortcuts you won't accept
✅ Check that the work actually happened
✅ Try to cheat it yourself first
✅ Test it on real traffic
If you can beat it without
