N+1 Blog

The Browser Automation Ecosystem for AI Agents: CLIs, Cloud Browsers, and AI-Native QA

Sun, 05 Jul 2026 00:00:00 +0000

TL;DR

Playwright CLI made a bet back in April 2026: stop streaming DOM/accessibility trees into an agent’s context window through MCP tool calls, and instead persist browser state to disk as compact YAML that the agent reads on demand. Three months on, Vercel Labs independently made the identical bet with agent-browser. That’s not a coincidence — it’s the shape the whole ecosystem is converging on. This post maps where every other piece fits: cloud browser infra, AI-native test reporting, and the consumer-facing agentic browsers that are a different animal entirely.

Introduction

My previous post on Playwright + Claude Code tracked one specific transition — Playwright’s MCP server giving way to a CLI-first approach for coding agents. That was useful in isolation, but it’s one tool in a much bigger space. Since then I kept running into adjacent tools solving overlapping problems: Vercel shipping their own browser CLI, Anthropic pushing Claude for Chrome to GA, and a wave of “AI-native” test reporting startups layering LLM analysis on top of Playwright/Cypress CI runs.

This post is a landscape map, not a tutorial. Three categories, what each tool actually is, and — because a lot of the marketing in this space overlaps — a decision guide for which one you actually reach for.

The three categories

flowchart TD A["Coding agent
(Claude Code, Cursor, etc.)"] subgraph L["1. Automation primitives"] PC["Playwright CLI"] AB["agent-browser (Vercel Labs)"] SH["Stagehand"] BU["Browser Use"] SK["Skyvern"] CLOUD["optional cloud execution:
Browserbase, Browser Use Cloud,
Vercel Sandbox"] CUR["Cursor Cloud Agents
(coding agent, not automation infra)"] end subgraph P["2. Consumer / enterprise agentic browsers"] CC["Claude for Chrome"] OP["OpenAI Operator"] ED["Edge Copilot Mode"] end subgraph Q["3. AI-native QA reporting"] RP["ReportPortal"] TD["TestDino"] CU["Currents.dev"] AT["Allure TestOps"] TO["Testomat.io"] end A --> L L -->|"CI results feed into"| Q L -.->|"same underlying mechanics,
different execution context"| P

1. Automation primitives — “how does an agent drive a browser”

This is the layer your coding agent actually calls.

Playwright CLI

Playwright CLI — Microsoft’s CLI-first tool, covered in depth in my earlier post. Since then: run-code now accepts a file argument instead of only inline snippets, show --annotate adds visual/structural annotations for agents, a new drop command handles drag-and-drop, and the CLI self-checks for stale installed Skills. Worth correcting my own record here: Playwright’s MCP server is not deprecated — Microsoft still recommends it for exploratory, self-healing, long-running agentic sessions that benefit from persistent state. CLI+Skills is the pick for high-throughput coding-agent workflows where token budget dominates. ~11.8k stars, 235 dependent projects — real adoption, still smaller than the OSS agent-browser frameworks below.

What it actually feels like to use

Setup is two commands — no .mcp.json, no server process to keep alive:

npm install -g @playwright/cli@latest
playwright-cli install --skills

That second command drops a Skill into .claude/skills/ (or the equivalent for Copilot/Cursor/Codex), so the agent discovers every command by reading playwright-cli --help on its own — no tool schema to hand-wire into context.

You can drive it by hand first, the same way the agent will:

playwright-cli open https://demo.playwright.dev/todomvc/ --headed
playwright-cli type "Buy groceries"
playwright-cli press Enter
playwright-cli type "Water flowers"
playwright-cli press Enter
playwright-cli check e21
playwright-cli screenshot

The part that actually changes how an agent works is what comes back after every command — not a blob of accessibility-tree JSON stuffed into the response, but a pointer to a YAML file on disk:

> playwright-cli goto https://demo.playwright.dev/todomvc/
### Page
- Page URL: https://demo.playwright.dev/todomvc/
- Page Title: TodoMVC
### Snapshot
[Snapshot](.playwright-cli/page-2026-02-14T19-22-42-679Z.yml)

The agent only opens that file when it actually needs to reason about the DOM — e.g. to find the e21 ref for a checkbox — instead of re-parsing a full tree on every single turn. That’s the whole “state on disk, not tool schemas in context” bet from the TL;DR, made concrete: ten actions in a row cost ten cheap CLI invocations plus however many snapshot reads the agent actually decides it needs, not ten full-page trees round-tripped through the model.

In practice, an end-to-end run from Claude Code looks like this:

> Use playwright skills to test the "add todo" flow on
  https://demo.playwright.dev/todomvc, then write a Playwright test for it.

Claude works it like a person would — open the page, snapshot to get refs, type and submit two todos, check one off, screenshot as evidence — and only after the manual walkthrough succeeds does it emit the automated test:

playwright-cli run-code --filename=verify.ts   # sanity-check a snippet before committing to it
playwright-cli generate-locator e21            # turn a ref into a stable Playwright locator

import { test, expect } from "@playwright/test";

test("can add and complete a todo", async ({ page }) => {
  await page.goto("https://demo.playwright.dev/todomvc/");
  await page.getByPlaceholder("What needs to be done?").fill("Buy groceries");
  await page.getByPlaceholder("What needs to be done?").press("Enter");
  await page.getByPlaceholder("What needs to be done?").fill("Water flowers");
  await page.getByPlaceholder("What needs to be done?").press("Enter");

  await page.getByRole("listitem").filter({ hasText: "Buy groceries" })
    .getByRole("checkbox").check();

  await expect(page.getByRole("listitem").filter({ hasText: "Buy groceries" }))
    .toHaveClass(/completed/);
});

Same manual-exploration-then-generate-test shape as the MCP workflow in my earlier post — the difference isn’t the workflow, it’s that every step along the way is a lightweight CLI call and a snapshot pointer instead of a tool-call response carrying the accessibility tree. If something misbehaves mid-run, playwright-cli show opens a live dashboard with a screencast of the session — useful for actually watching what the agent is doing rather than reading a wall of tool-call logs after the fact.

agent-browser (Vercel Labs)

vercel-labs/agent-browser — a Rust CLI from Vercel Labs, same philosophy as Playwright CLI: accessibility-tree snapshots with persistent element refs (@e1, @e2) instead of re-serializing the DOM every turn.

Integrations: @agent-browser/sandbox for running itself inside ephemeral Vercel Sandbox microVMs, plus its own MCP server (agent-browser mcp) for tools that still expect one. Documented to work with Claude Code, Cursor, Codex, Copilot, Windsurf, Gemini CLI, Goose, and OpenCode.
Adoption: 37.9k stars — bigger than Playwright CLI, though that’s partly Vercel’s reach rather than a signal of deeper coding-agent adoption specifically. One credible independent write-up: Pulumi’s engineering blog used it with Claude Code in a self-verifying CI loop.

What it actually feels like to use

Setup mirrors Playwright CLI’s two-command simplicity:

npm install -g agent-browser
agent-browser install   # downloads Chrome for Testing on first run

Driving the same TodoMVC flow by hand, using refs from a snapshot instead of Playwright CLI’s e1/e2 style:

agent-browser open https://demo.playwright.dev/todomvc/
agent-browser snapshot

- textbox "What needs to be done?" [ref=e1]
- list "todo-list" [ref=e2]

agent-browser fill @e1 "Buy groceries"
agent-browser press Enter
agent-browser fill @e1 "Water flowers"
agent-browser press Enter
agent-browser snapshot
agent-browser check @e6
agent-browser screenshot todo.png

Same shape as the Playwright CLI session — snapshot, act on a ref, snapshot again — just with @e1-style refs instead of bare e1, and fill/press instead of type/press. The open on the first line pays a browser-launch cost once; everything after it is fast because a background daemon stays warm between commands. Under the hood that daemon is Rust talking straight to the Chrome DevTools Protocol (CDP — the interface Chrome/Chromium expose for controlling pages, dispatching input, reading network traffic), where Playwright’s own bindings reach the same protocol through a bundled Node.js driver process instead. It detects an existing Playwright or Puppeteer install to reuse the Chrome binary, but doesn’t depend on either at runtime.

Where it diverges from Playwright CLI is what happens once the exploration succeeds. Playwright CLI’s agent loop ends by emitting an @playwright/test file, because Playwright ships a full test runner. agent-browser doesn’t — there’s no test-runner equivalent — so the natural artifact is a repeatable script of the same CLI commands via batch, which also cuts the per-command process overhead:

agent-browser batch \
  '["open", "https://demo.playwright.dev/todomvc/"]' \
  '["fill", "@e1", "Buy groceries"]' \
  '["press", "Enter"]' \
  '["fill", "@e1", "Water flowers"]' \
  '["press", "Enter"]' \
  '["check", "@e6"]' \
  '["screenshot", "todo.png"]'

That’s the practical tradeoff versus Playwright CLI: a faster, dependency-free daemon and one CLI surface for both exploration and repeatable automation, in exchange for giving up Playwright’s mature test-runner ecosystem (fixtures, trace viewer, CI reporters) — you’d still reach for @playwright/test or Stagehand if you actually want a maintained regression suite out of the session.

Stagehand

Stagehand (Browserbase, MIT, 23.3k stars) — four primitives: act, extract, observe, agent. Where Playwright CLI and agent-browser are CLIs a coding agent shells out to, Stagehand is a TypeScript SDK you write code against directly — the LLM decision loop lives inside your own script, not behind a separate command-line process.

What it actually feels like to use

Setup is a scaffolding CLI, not a global install:

npx create-browser-app
cd my-stagehand-app
cp .env.example .env   # add OPENAI_API_KEY, BROWSERBASE_API_KEY, etc.
npm start

That scaffolds a runnable index.ts using the same four primitives. Adapted to the TodoMVC flow from the previous two sections:

import { Stagehand } from "@browserbasehq/stagehand";
import { z } from "zod";

const stagehand = new Stagehand({ env: "LOCAL" }); // or "BROWSERBASE" for a cloud session
await stagehand.init();

const page = stagehand.context.pages()[0];
await page.goto("https://demo.playwright.dev/todomvc/");

await stagehand.act("type 'Buy groceries' into the new-todo box and press Enter");
await stagehand.act("type 'Water flowers' into the new-todo box and press Enter");

const observeResult = await stagehand.observe("what can I click to mark the first todo as done?");
const actResult = await stagehand.act(observeResult[0]); // replay the previewed action, not a fresh prompt
console.log(actResult.cacheStatus); // "MISS" the first time, "HIT" on every rerun after

const { count } = await stagehand.extract(
  "how many completed todos are there?",
  z.object({ count: z.number() }),
);

observe() is the preview step — it returns a candidate action without running it, so you can inspect or approve it before act() executes it. Run the same script a second time and act() reports cacheStatus: "HIT": Browserbase’s server-side cache keys on the instruction, page content, and options, so a repeated call skips the LLM entirely and replays deterministically — until the page’s structure actually changes, at which point it falls back to a fresh model call automatically. That’s the “prompt-first while exploring, code-first once stable” pitch made concrete: the same three lines cost real inference on the first run and roughly zero on every rerun after, with no separate step to convert exploration into a locator.

For the same “just describe the outcome” shape as Playwright CLI’s and agent-browser’s agent-driven sessions, agent() wraps the whole flow:

const agent = stagehand.agent();
await agent.execute(
  "Add two todos, 'Buy groceries' and 'Water flowers', then mark the first one complete."
);

The competitive angle versus the CLI tools: Playwright CLI and agent-browser hand an agent raw commands and a snapshot file, and the agent decides what to do next every turn — there’s no built-in mechanism for skipping repeated LLM calls once a flow is known-good. Stagehand’s caching does that natively, and switching from a local Chromium to a cloud one is a one-line config change (env: "BROWSERBASE") rather than a separate sandbox package or infra wiring. Browserbase itself is managed headless Chromium at scale, plus a Search API (Exa-powered), a Fetch API, and “Agent Identity” — Free / Developer $20/mo / Startup $99/mo / custom Scale, backed by a $40M Series B at ~$300M valuation.

Browser Use

Browser Use (MIT, Python, 102.8k GitHub stars as of writing — actively maintained, latest release 0.13.3, July 2026) takes a different shape than everything above it. Where Stagehand exposes act/extract/observe as primitives you compose yourself, Browser Use is fully agentic by default: you hand it one task string and an Agent.run() loop drives the whole session end-to-end, closer in spirit to Skyvern than to Stagehand.

What it actually feels like to use

Install is a package, not a CLI-first tool like the previous two sections:

uv add browser-use   # or: pip install browser-use

Requires Python 3.11+. The minimal shape is a single Agent, not a sequence of act/type/click calls:

import asyncio
from browser_use import Agent, BrowserProfile, ChatBrowserUse

async def main():
    agent = Agent(
        task=(
            "Go to https://demo.playwright.dev/todomvc/, add two todos "
            "'Buy groceries' and 'Water flowers', then mark the first one complete."
        ),
        llm=ChatBrowserUse(model="openai/gpt-5.5"),
        browser_profile=BrowserProfile(headless=False),
    )
    history = await agent.run()
    print(history.final_result())

asyncio.run(main())

Compare that to the Stagehand agent() call covering the identical task in the previous section — same one-sentence-task shape, but there’s no lower-level act/observe escape hatch to drop into if you want a specific step to be deterministic instead of re-planned by the model. Extensibility works the other direction: you register custom Tools() via @tools.action decorators that the agent can call mid-run (e.g. a “read this file” or “post to Slack” action), rather than scripting page-level steps yourself.

There’s also a scaffolding command, same idea as Stagehand’s create-browser-app:

uvx browser-use init --template default   # writes a runnable browser_use_default.py

And a separate CLI surface (browser-use, powered by a companion project called Browser Harness) that drops you into a scripted REPL for direct page control rather than task-driven automation — plus a browser-use skill command that registers itself as a Claude Code/Codex Skill, letting a coding agent drive a connected browser directly instead of going through the Python Agent class at all.

The competitive pitch: multi-provider LLM support out of the box (ChatBrowserUse proxies OpenAI, Anthropic, and Google models behind one BROWSER_USE_API_KEY, plus direct ChatOpenAI/ChatAnthropic classes and local Ollama models), and a company-run benchmark (browser-use/benchmark, “BU Bench V1”) comparing model success rates — the project is as much a benchmarking/model-optimization effort as a browser-automation library. Where it’s genuinely thinner than Stagehand: no documented cache-and-replay mechanism, so every run costs a full agentic loop of LLM calls rather than a cheap cache hit on repeat.

Unlike Stagehand’s one-line env: "BROWSERBASE" swap, cloud isn’t a constructor flag on the same Agent class here — Browser Use Cloud (cloud.browser-use.com) is a separate hosted product: submit a natural-language task over the API, it spins up a remote browser and agent, and gives you a live, watchable session URL. Pay-as-you-go from $0.06/browser-hour, or $100/mo Starter.

Skyvern

Skyvern (AGPL-3.0, Python, 22.1k stars) is the odd one out in this category: not a CLI, not a library you import and drive yourself, but a local service — a FastAPI server plus a bundled web UI — that you talk to through a Python/TS SDK or the UI directly. Its pitch is vision-LLM-assisted form-filling at scale (insurance quotes, government forms, checkout flows), with a hybrid grounding model rather than a pure no-selectors one: you can drive it with plain CSS/XPath, a natural-language prompt, or “AI fallback” (selector first, AI only on failure).

What it actually feels like to use

Quickstart spins up the whole service, not just a package:

pip install "skyvern[all]"
skyvern quickstart   # starts the FastAPI server + UI at http://localhost:8080

The SDK then layers Playwright-level control and AI-augmented calls on the same page object — the interesting part is that they compose, so you choose per-step how much AI you want:

from skyvern import Skyvern

skyvern = Skyvern.local()  # or Skyvern(api_key=...) for Skyvern Cloud
browser = await skyvern.launch_cloud_browser()
page = await browser.get_working_page()

await page.goto("https://demo.playwright.dev/todomvc/")
await page.click(prompt="the new-todo input box")          # AI-located, no selector written
await page.agent.run_task(                                 # or hand off the whole flow as one task
    "Add two todos, 'Buy groceries' and 'Water flowers', then mark the first one complete."
)
result = await page.extract(
    "how many completed todos are there?", schema={"count": "int"}
)

page.act(prompt) / page.extract(prompt, schema) / page.validate(prompt) are the Stagehand-shaped primitives here; page.agent.run_task(...) is the Browser-Use-shaped one-liner. page.agent.run_workflow(workflow_id) invokes a saved, structured multi-step definition built in Skyvern’s no-code workflow builder (Browser Task/Action blocks, data extraction, loops, HTTP requests, custom code) — the closest thing here to a “record once, replay forever” artifact, though it’s explicitly authored rather than auto-generated from a successful AI run.

Skyvern’s blog/technical-report material references a “Route Memorization” feature that auto-converts a successful AI path into a deterministic Playwright script — not documented in the README, so treat it as unconfirmed rather than a shipped feature.

The genuinely distinctive bits versus Browser Use and Stagehand: first-class 2FA/TOTP handling (QR, email, SMS) with dedicated docs, password-manager integration (Bitwarden today, 1Password/LastPass planned), an MCP server, and Zapier/Make/N8N integrations — this is a tool built around unattended, credentialed form-filling rather than exploratory browsing. Its 85.85% WebVoyager score is self-reported by Skyvern; a neutral aggregator (Steel’s leaderboard) repeats the same number, but that’s a compilation of vendor-reported runs, not an independent re-execution — treat it as directionally useful, not a controlled benchmark.

Cloud is Skyvern Cloud (Skyvern(api_key=...) instead of Skyvern.local()) — the README is explicit that the OSS core is feature-complete except for what it calls anti-bot measures: proxy rotation and CAPTCHA solving are cloud-exclusive, alongside no-infra hosting and parallel run capacity.

Cursor Cloud Agents

A different shape of “cloud browser” than everything above: Cursor Cloud Agents (cursor.com/cloud) isn’t infra you point a browser-automation library at — it’s a coding-agent product that happens to bundle a browser, not a standalone automation service like Browserbase or Browser Use Cloud.

You dispatch a task from wherever you already work, not just the IDE: Cursor Desktop’s “Cloud” option, the web dashboard at cursor.com/agents, the mobile apps, an @cursor mention on a GitHub/GitLab/Bitbucket PR comment, Slack, Teams, a Linear/Jira assignment, or the API. Each task gets its own ephemeral cloud VM — full desktop and terminal, not just a headless container — that clones the repo onto a working branch, installs dependencies (agent-led setup, a saved snapshot, or a custom Dockerfile via .cursor/environment.json), and gets network access. Nothing is streamed to you live by default: a dashboard lists running agents, and completion produces a merge-ready PR plus a video recording of the agent’s actual desktop/browser session, screenshots, and logs — though you can grab remote-desktop control of a still-running VM if you want to watch or intervene.

The browser-verification part isn’t a flag you set — it’s the agent deciding, mid-task, to open the app it just changed and click through it to check the result, then attaching that session as evidence rather than handing off its first untested attempt. Cursor’s own launch post claimed 30% of their internally merged PRs came from agents running this way; CEO Michael Truell has since cited 35%+ in press coverage as adoption grew. Billing is API/token-metered per model rather than a separate SKU — cloud agents always run in Max Mode, gated by a spend limit you set once, and the capability itself is bundled into Ultra/Teams/Enterprise plans.

2. Consumer / enterprise agentic browsers — same tech, different job

These use the same LLM-drives-a-browser mechanics as category 1, but the split holds up: they act inside your authenticated browser session — your email, your accounts, your cookies — not a disposable automation instance, and they ship as closed products from a single vendor rather than a library you wire into your own agent loop.

Claude for Chrome

A Chrome extension, not a separate app — install it from the Chrome Web Store and it acts inside your existing logged-in tabs. Still in beta (Anthropic’s own product page calls it “a beta feature with unique risks”), available on all paid plans rather than gated to a specific tier. Two concrete capabilities beyond one-shot “do this now” prompts: scheduled tasks that re-run on a daily/weekly/monthly cadence, and multi-tab workflows where you drag tabs into a Claude tab group so the agent can read and act across all of them at once.

The programmable primitive underneath, if you want to build the same click/type/screenshot loop yourself rather than use the extension, is Anthropic’s Computer Use tool on the Developer Platform (also available via Bedrock and Vertex): the model returns a tool_use action — screenshot, left_click, type, scroll — as JSON, your code executes it against a VM/container you run, and you feed the resulting screenshot back in.

Worth knowing: CyberScoop reported a real vulnerability chain — a permissive *.claude.ai origin allowlist combined with a DOM XSS bug in a CAPTCHA component let another browser extension achieve zero-click prompt injection and hijack the agent’s session. Disclosed December 2025, patched in extension v1.0.41. This class of risk (an agent that can read and act on any page you have open) is structural to the category, not unique to Anthropic’s implementation — expect similar disclosures from other in-browser agents as they mature.

OpenAI Operator / ChatGPT agent

Folded into ChatGPT’s unified agent mode. Third-party reviews through 2026 have been mixed to critical (overcautious, constant confirmation prompts) — treat that sentiment as opinion-blog territory, not a controlled study.

Microsoft Edge Copilot Mode

Enterprise/IT-policy-gated agentic browsing for Edge for Business, still in limited preview. Notable mainly as market context: Microsoft is betting on agentic browsing at the OS/browser level with Edge, while separately shipping Playwright CLI as the dev-tool-layer bet — same company, two different audiences.

One neutral cross-tool reference point, if you want actual numbers instead of scattered vendor claims: Steel’s open leaderboard tracks Skyvern, Browser Use, Operator, and Claude Computer Use against WebVoyager/OSWorld/Online-Mind2Web. It’s still self-reported per source, dated and attributed rather than independently re-run, and the maintainers themselves flag that WebVoyager has saturated (>90% scores common at the top) — useful for rough ordering, not a rigorous head-to-head.

3. AI-native QA reporting

These tools don’t drive a browser. They consume Playwright/Cypress CI output and add an AI layer on top — failure classification, flakiness scoring, root-cause tagging. Easy to conflate with category 1 because they’re all “AI + Playwright,” but they solve “why did this test fail” rather than “how do I drive this test.”

ReportPortal

ReportPortal — open-source (Apache-2.0, EPAM-originated), self-hosted-first test analytics platform, and the most technically substantive AI story of the group: its service-auto-analyzer component runs similarity/clustering search over indexed logs and stack traces (Elasticsearch-backed) to auto-assign new failures to previously triaged defect types — genuine ML/information-retrieval, not an LLM wrapper. Ships an official MCP server (reportportal/reportportal-mcp-server) exposing auto-analysis, unique-error-analysis, quality gates, and launch/log queries to any MCP-speaking agent. Free to self-host indefinitely — the only tool in this group without a paywall between you and the AI features.

TestDino

TestDino — AI failure classification into four buckets (actual bug / UI-change failure / flaky / misc) with confidence scores, per-test stability tracking, GitHub PR annotations. The one thing that separates it from older reporting tools: a genuine open-source MCP server with 27 tools letting Claude/Cursor/Copilot query test-run and root-cause data directly — confirmed as a real engineering artifact, not just marketing copy, though I found no independent user reviews (G2/Reddit/HN) corroborating how well it works in practice.

Currents.dev

Currents.dev — a Playwright/Cypress-focused CI dashboard and test orchestrator: aggregates run results, auto-balances specs across CI machines (claiming up to ~50% faster than native Playwright sharding), flaky-test detection, and analytics/regression trends. Its AI story is an official MCP server plus a Playwright “agent skill” and IDE extension, letting Claude/Cursor/Copilot query historical test data directly — the flaky-detection and error-classification mechanisms themselves aren’t documented as ML- or LLM-based, so treat “AI-driven” there as marketing rather than a verified technique.

Allure TestOps

Allure TestOps (Qameta) — the paid layer on top of the widely-used open-source Allure Report standard: manual + automation test management, CI/BTS/TMS integrations, high-load result storage. Its AI story is younger and narrower than ReportPortal’s — no native ML failure analysis — built instead around an official MCP server (public beta since release 26.1.1, ~13 tools) for creating/searching test cases via its query language and managing launches, results, and test plans from Claude Desktop, Claude Code, or Cursor.

Testomat.io

Testomat.io — a manual + automated test-management tool built around AI throughout the workflow rather than one bolt-on feature: AI Failure Clusterization (groups automated failures by detected pattern, shipped on all tiers including free — older claims that it “lacks AI analysis” are stale) and an embedded AI Agent Test Assistant. No first-party MCP server surfaced as of this research, so its AI integration is in-product rather than agent-facing.

Feature and pricing comparison, verified directly against each vendor’s own site/docs/pricing page:

Tool	AI/ML mechanism	Agent integration	Entry price	Notes
ReportPortal	ML/similarity clustering (Elasticsearch-backed auto-analysis)	Official MCP server (Docker, stdio/HTTP)	Free, self-hosted (Apache-2.0)	SaaS also available, ~$570–600/mo (Startup tier), if you don't want to self-host
TestDino	4-bucket failure classification with confidence scores	Open-source MCP server, 27 tools	$39/mo (annual) / $49/mo (monthly)	Free tier: 5k executions/mo
Currents.dev	Undocumented mechanism (marketed as "AI-driven" flaky detection)	Official MCP server + Playwright agent skill	$49/mo	Usage-based, 10k results included, $5/1k overage
Allure TestOps	None native — AI story is the MCP server, not ML analysis	Official MCP server, public beta, ~13 tools	$39/user/mo, sliding to $30/user/mo at scale	Per-seat, not per-execution
Testomat.io	AI Failure Clusterization + AI Agent Test Assistant	None confirmed	$30/mo/user ($27 annual)	AI clusterization ships on all tiers, including free

None of these generate or drive tests — they’re analytics/reporting layers on top of whatever ran the browser in category 1.

Decision guide

If you're...	Reach for
Testing/exploring a web app from Claude Code, token budget matters	Playwright CLI or `agent-browser`
Running the automation itself inside ephemeral Vercel Sandbox microVMs	`agent-browser`
Want natural-language actions with fallback to deterministic code, plus caching	Stagehand
Want one task string to drive the whole session instead of composing primitives	Browser Use
Automating credentialed, form-heavy flows at scale (2FA, password managers)	Skyvern
Need production-grade cloud browser sessions behind an API	Browserbase
Want a coding agent that verifies its own change by driving a browser, fully in the cloud	Cursor Cloud Agents
Want AI triage on top of existing Playwright/Cypress CI failures	ReportPortal — free, self-hosted, and the most substantive ML story (TestDino/Currents/Allure otherwise, if you'd rather pay for SaaS than run infra)
Want a human-facing agentic browser, not a coding-agent tool	Claude for Chrome

Conclusion

The clearest signal in this space right now isn’t any single tool — it’s that Playwright CLI’s and Vercel’s agent-browser independently converged on the same architecture: state on disk, not tool schemas in context. That’s a strong sign the MCP-tool-call pattern for driving a browser is being displaced by CLI-first design specifically for coding agents, even as MCP stays relevant for other integration points.

Category 1 itself splits further than “primitives” implies: Playwright CLI, agent-browser, and Stagehand hand you building blocks to compose; Browser Use and Skyvern go further and take a single task string, running the whole agentic loop themselves. That’s a real architectural choice, not a maturity gradient — which one you want depends on whether you need a specific step to stay deterministic or you’re fine with the model re-planning every turn.

There’s a second, almost inverse convergence in category 3: TestDino, Currents.dev, ReportPortal, and Allure TestOps are all independently shipping MCP servers, but not to drive anything — they let an agent query test-run and root-cause data instead. Opposite job from category 1’s story, same protocol. Everything else in this post — consumer agentic browsers — sits in an adjacent lane that’s easy to lump in under “AI browser tools” but solves a different problem: acting inside your session rather than a disposable automation instance.

References

Mapping the AI Context & Memory Ecosystem: context-mode, graphify, cognee, OpenMetadata

Sat, 04 Jul 2026 00:00:00 +0000

TL;DR: Four repos, two markets that turn out to share one primitive — the graph. context-mode and graphify compress/structure what an agent sees in a single session; cognee builds persistent cross-session memory on the same graph-RAG substrate; OpenMetadata is the established data-catalog platform now repositioning as a “context layer for AI.”

Full report: ai-context-ecosystem report →

Why this report

Every one of these tools claims to solve “the agent doesn’t remember/see enough” problem, but they attack it from different layers — one CLI session, one agent’s memory store, or an entire org’s data warehouse. It wasn’t obvious from the READMEs alone whether any of them actually compete with each other, so I put together a working scratchpad with the scratch CLI: one file per repo and per competitor category, a cross-cutting-gaps section, and a use-cases section built from web research rather than invented scenarios.

tmux-message-bus: Giving Claude Code Instances a Mailbox

Sat, 27 Jun 2026 00:00:00 +0000

TL;DR: tmux-message-bus is an experiment: a durable, ordered, at-least-once message bus. It lets independent Claude Code instances, each in its own tmux window or session, hand work to each other and trust it arrives. The filesystem (one SQLite-WAL database) is the transport; send-keys is demoted to an optional doorbell. The whole thing is built out of Claude Code’s lifecycle hooks, a small CLI, and config. There’s no programmatic hook into the agent itself. This post walks through how that works.

Source: github.com/nikiforovall/tmux-message-bus

Two ways to make a coding agent programmable

If you want to extend a terminal coding agent today, there are broadly two models on offer.

The first is an imperative API into the agent itself. pi is the clearest example: extensions are TypeScript modules, loaded without a compile step, that “hook directly into the agent’s reasoning loop, event lifecycle, and terminal UI.” You register tools the LLM can call, subscribe to lifecycle events that fire inside the agent loop, rewrite the compaction pipeline, gate tool calls, and the LLM never sees any of it. opencode takes a similar shape from a different angle: it runs a headless HTTP server with an OpenAPI spec and ships an official @opencode-ai/sdk, plus a plugin system where each plugin is a TS module exporting hook objects. Either way, you’re writing code that runs in-process, with first-class access to the agent’s internals.

The second model is declarative composition from the outside. This is Claude Code. You get lifecycle hooks (shell commands that fire on events), plugins, skills, MCP servers, and a CLI. There is no SDK that drops you inside the reasoning loop. A hook is a process the harness spawns at a defined moment; it reads JSON on stdin and can return a small structured decision on stdout. That’s the whole contract. It’s lower-level and language-agnostic, and it never lets you write code that the agent runs as itself.

The imperative model is more ergonomic, no question. Whether it’s more powerful is less obvious, and anyway with Claude Code you don’t get to pick: the declarative kit is what’s on offer. So the honest question is how much you can actually build with it. I went looking for a problem that would stress it.

The problem: agents are islands

Open three Claude Code sessions in tmux and you have three islands. There is no clean way for one to hand a task to another and know it landed. Claude Code’s native multi-agent tools — Agent, SendMessage, the Task tools — only address subagents within a single process’s tree. They cannot reach an independent claude running in another tmux window. Nothing native crosses that boundary.

The common workaround is to script tmux send-keys to type into another pane’s input box. It’s fragile in exactly the way you’d expect: the receiver is an Ink/React TUI in nondeterministic state, so your keystrokes collide with streaming output, permission prompts, and autocomplete. There’s no acknowledgement, no ordering, no durability. If the target window moved or the agent is mid-turn, the message is just gone.

So I wanted a real mailbox: durable, ordered, at-least-once delivery between agents that never have to be awake at the same time. In the imperative model you might build this as a plugin holding a socket. Here, there’s nowhere to hold a socket. There’s no long-lived process I control. There are only hooks.

Hooks as primitives

The trick is to stop thinking of hooks as callbacks and start thinking of them as primitives you compose a system out of. Claude Code fires hooks at distinct lifecycle moments, and each one maps onto a piece of a message bus. SessionStart registers an instance: a hook writes it into a shared registry (who I am, my pid, my current pane), and sweeps stale state while it’s there. Stop delivers: when an agent finishes a turn, a hook drains its mailbox, claiming new messages, injecting them, marking them done. UserPromptSubmit handles the doorbell wake, which I’ll get to below. SessionEnd garbage-collects on graceful exit, keeping the database bounded.

None of these knows about a “bus.” Each is a few lines of shell calling a small Node CLI that owns the SQLite database. The bus is what emerges when you wire these events to a shared store. It’s assembled out of lifecycle events rather than coded into a runtime.

flowchart LR subgraph hostA["tmux window A"] A["Agent A
(Claude instance)"] end subgraph hostB["tmux window B"] B["Agent B
(Claude instance)"] end DB[("bus.db
SQLite WAL
agents + messages")] A -- "1 send (durable INSERT)" --> DB A -. "2 doorbell: send-keys «bus»
(best-effort wake)" .-> B B -- "3 claim → act → ack" --> DB DB -. registry / liveness .- A DB -. registry / liveness .- B

The one genuinely load-bearing trick lives in the Stop hook. A Stop hook can return {decision: "block", reason: "..."}, which tells the harness not to stop yet, with the reason supplied. That re-prompt is the delivery mechanism. When an agent tries to end its turn, the hook claims any waiting mail and feeds the bodies back as the reason, so the agent keeps going and acts on them. A clear-on-read claim means the next Stop sees an empty mailbox and is allowed to terminate, so there’s no infinite loop. Everything else is plumbing around that one observed behavior.

Two ideas that make it robust

Building on top of a TUI you don’t control forces some discipline. Two distributed-systems ideas did the heavy lifting.

The first is to separate delivery from notification. Sending a message is a durable INSERT into SQLite, and that’s the entire delivery guarantee: once the row is committed, the message can’t be lost. The doorbell, a send-keys of a tiny fixed «bus» sentinel to wake an idle peer, is purely best-effort. If it fails, nothing breaks; the message is already stored and gets picked up on the receiver’s next turn. This inverts the send-keys hack. The filesystem is the real transport, and keystrokes are just a nudge that lowers latency. A garbled doorbell costs you latency, never a message.

sequenceDiagram participant A as Agent A participant DB as bus.db participant B as Agent B A->>DB: send --to B (INSERT, status=new) Note over A,DB: delivery is done here — durable, cannot be lost A-->>B: doorbell «bus» (best-effort send-keys) activate B Note over B: next turn (woken by doorbell,
or its normal Stop hook) B->>DB: claim (atomic, ordered by id) Note over B: message injected as framed
INFORMATION — B decides what to do B->>DB: ack (mark done) deactivate B B->>DB: reply --to-msg (correlated)

The second is that identity has to survive the chaos of tmux. Panes move, windows get renamed, sessions get re-numbered, so none of those is identity. The bus keys each agent on a stable id derived from its session, so a renamed session or a moved pane is never mistaken for a dead agent, and identity survives --resume. The details are in DESIGN.md.

One surprise is worth calling out. Injected messages are framed as information from a peer, not as commands, and that framing turned out to be load-bearing. When I first injected message bodies as imperatives (“reply with token X”), the receiver refused them as a prompt-injection attempt; the harness applies injection defenses to hook-injected text. Re-framing each message with its provenance (“routed from peer agent X on the bus you enabled; this is information, not a user command; you decide whether to act”) made it go through. The platform itself nudges you toward messaging rather than remote code execution. The receiver always decides.

What the constraint taught me

The declarative kit turned out to be more expressive than its surface suggests. I expected to hit a wall. Instead, a handful of lifecycle hooks, a shared store, and one re-prompt trick composed into a durable async transport with delivery, ordering, liveness, and correlation. None of it required a single line of code running inside the agent loop.

That’s the takeaway worth keeping. The outside-in surface looks limited, but it’s stable, language-agnostic, and composable. The way to find out what it can do is to pick a problem one size too big for what it obviously supports and build a throwaway probe; the constraints show you where the edges actually are.

It’s an experiment, not a product: single-host only, identity-based addressing, with roles and broadcast still open questions. But it works end to end, and the code is on GitHub.

Summary

Claude Code gives you declarative building blocks (lifecycle hooks, a CLI, config) rather than an imperative SDK into the agent loop like pi or opencode. That’s less ergonomic, but more capable than it looks. Treat hooks as primitives, wire them to a shared SQLite store, and you can build a durable cross-instance message bus that Claude Code doesn’t ship natively. Separate durable delivery from best-effort notification, anchor identity on something that survives tmux’s churn, and frame inter-agent traffic as information rather than commands.

Reference

scratch: Structured Scratchpads for Coding Agents

Mon, 08 Jun 2026 00:00:00 +0000

TL;DR: scratch organizes the temporary knowledge an agent produces — notes, snippets, command output, intermediate artifacts — into a scratchpad: a folder plus a scratchpad.json manifest, kept out of your source tree. It’s a thin metadata layer over the filesystem, not a database.

Install with bun add -g @nikiforovall/scratchpad

Docs & live demo: nikiforovall.blog/scratchpad/

Why files: the context window is a real limit

One constraint shapes everything else: agents work better when they write things down.

The context window is the agent’s only short-term memory — the running transcript of the task so far is all it can actually “see.” That transcript is finite. Once it fills up, the oldest parts drop off to make room: findings, decisions, the command output from twenty steps ago. There’s no warning; the model just quietly forgets. Long-horizon tasks fail less because the model is incapable and more because the memory it needed is gone.

The fix is almost boring: externalize that memory to the filesystem. Write the plan to a .md file. Write findings to a .md file. Now the agent re-reads exactly what it needs, when it needs it, instead of hoping it’s still in context — and a markdown file is durable, greppable, diffable, and survives a compaction. “Markdown files are all you need” is becoming a real pattern, not a hack.

So agents should write to disk. The open question is what happens to those files next.

The problem: temporary agent knowledge has nowhere to go

Spend a session with a coding agent and watch what it produces. It greps the codebase and finds five relevant files. It writes a quick design note before touching code. It runs the test suite and the failure output is what it reasons about next. It sketches a mermaid diagram to make sense of a flow.

All of that is real, useful work. And almost none of it sticks around.

Agents generate a lot of temporary knowledge per session, and nothing keeps track of it. It ends up scattered across the repo, buried in chat history, or lost when the context window rolls over.

The knowledge is temporary — tied to a session or a task, not to the project forever — but “temporary” shouldn’t mean “lost” or “scattered.”

The solution: a scratchpad is just a folder

A scratchpad is a folder containing a scratchpad.json manifest. The folder path is its identity — there’s no central store, no daemon, no SQLite file hidden away under your config. scratch is a thin metadata layer over the filesystem: it creates pads, registers files with a description and type, and shows you what’s there.

The agent writes files with its normal tools. Then it registers each one:

scratch new "auth-refactor" --dir _scratchpads
# → creates _scratchpads/auth-refactor/ + scratchpad.json

# agent writes _scratchpads/auth-refactor/findings.md with its normal tools, then:
scratch add "auth-refactor" findings.md \
  --desc "where the current token flow breaks" \
  --type note --tag auth,investigation

The --desc is the point: it captures why this file exists, not just that it does. Months later — or two compactions later — that one line is the difference between a useful artifact and a mystery file.

Two things make this pleasant in practice. First, no lock-in: it’s files on disk. Delete the folder and the knowledge is gone; the CLI never authors, copies, or moves your content. Second, a human can actually read it.

The demo: see what the agent gathered

Run scratch ui and you get a read-only, two-pane viewer in a “Lab Notebook” theme that auto-detects light/dark:

Markdown is rendered (with a raw/rendered toggle), code is syntax-highlighted, mermaid diagrams draw, images show inline. It opens a native window by default and falls back to your browser if the native backend isn’t available.

Scratchpads are also shareable. scratch export bundles a whole pad into a single self-contained HTML file — file contents embedded, no server needed — that you can hand to a teammate or link from anywhere. Here’s one exported live:

Try it: a scratchpad, exported →

No digging through transcripts to find what the agent learned. You open the viewer — or the exported file — and browse.

Wiring it into Claude Code

The repo doubles as a Claude Code plugin marketplace. Add it and the agent gets a scratch skill that teaches it when and how to drive the CLI — so it reaches for a scratchpad on its own when a task starts generating keepable knowledge:

/plugin marketplace add NikiforovAll/scratchpad
/plugin install scratchpad@scratchpad

The plugin ships the skill; the scratch CLI itself still comes from the install above:

bun add -g @nikiforovall/scratchpad

From there the agent runs the loop — create, write, register, browse — without you micromanaging it.

Planning with scratchpads

The plugin ships a second skill, planning-with-scratchpad, that puts a real workflow on top of the raw CLI. Its goal: use scratchpads as external memory for complex, multi-session tasks — the kind where a single Plan Mode pass isn’t enough and you need research, decisions, and rationale to survive context limits and session boundaries.

The convention is one pad per task under _plans/, and one concern per file rather than a sprawling 200-line plan.md:

plan.md — the index: goal, phases, status, errors. Kept lean; it’s the map, not the territory.
research-<topic>.md — sources and findings, one file per distinct topic.
decisions.md — options, the choice, and why (the rationale future-you needs).
scratch-<label>.md — disposable working notes and prototypes.

The flow it encourages: research into files first, outline every phase and task in plan.md and get your approval before any code is written, then batch-create the tasks. Because every file is registered with scratch, you open scratch ui and watch the plan take shape in the viewer — the agent’s thinking laid out as browsable files instead of buried in a long transcript. You can read the plan and steer it before it turns into code.

▶ Show the full planning-with-scratchpad SKILL.md (click to expand)

---
name: planning-with-scratchpad
description: Use when user explicitly requests planning with a scratchpad, or asks for persistent tracking of a complex task that the human can browse visually. Backs planning files with the `scratch` CLI so a pad's files are registered and viewable.
disable-model-invocation: true
---

# Planning with Scratchpad

Use persistent markdown files as external memory for complex tasks. Files survive context limits and session boundaries. The files live in a **scratchpad** (a `_plans/` folder + manifest) so the human can browse the plan in the visual viewer.

**Load the `scratch` skill first** — it owns all CLI mechanics (`new`, `add`, `ls`, `show`, `ui`). This skill adds the planning conventions on top.

Use persistent files for planning, `TaskCreate` for tracking execution. Before closing session, use `AskUserQuestion` to confirm findings, decisions, and deliverables are satisfactory.

## Bundled References

Read these when you need deeper guidance on a specific aspect:

- **[reference.md](reference.md)** — Context engineering principles (Manus-inspired). Read when designing how to structure planning files for a long-running or multi-agent task.
- **[examples.md](examples.md)** — Real task examples showing different file combinations. Read when unsure which files to create for a given task shape.

## When to Use

- User explicitly requests planning/tracking with a scratchpad
- Complex multi-session tasks
- Research-heavy work requiring persistent notes
- Tasks where decisions and rationale need to be preserved

## When NOT to Use

- Simple single-session tasks (use Plan Mode instead)
- Quick fixes or small changes
- Tasks with clear requirements needing no research

## Directory Convention

One pad per task, created in `_plans/` with a date-prefixed folder:

```
scratch new "<task-name>" --dir _plans --id "$SESSION_ID"
```

This creates `_plans/YYYY-MM-DD-<slug>/`. Write files into that folder with your normal tools, then **register each** with `scratch add` so it appears in the viewer:

```
_plans/
  2026-01-08-dark-mode-toggle/
    scratchpad.json          # manifest (managed by scratch)
    plan.md
    research-css-strategies.md
    decisions.md
  2026-01-09-api-auth-refactor/
    scratchpad.json
    plan.md
    research-auth-providers.md
    research-token-storage.md
    decision-oauth-vs-saml.md
    decision-session-strategy.md
    references.md
    scratch-migration-steps.md
```

**Naming:** `YYYY-MM-DD-task-name` (kebab-case, concise description) — `scratch new` derives this from the task name + date.

## File Types — One Concern Per File

Each file owns a single concern. Never merge concerns into one file — a 200-line plan.md that also contains research notes, decisions, and scratch work is hard to navigate and easy to lose track of. Split early; you can always cross-reference with relative links. Register each file with the matching `scratch add --type`.

| File                  | Concern                                      | Create when                                                                                                         | `--type`    |
| --------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | ----------- |
| `plan.md`             | Goal, phases, status, errors                 | Always — this is the index                                                                                          | `note`      |
| `research-<topic>.md` | Sources, findings for one topic              | Any research needed. One file per distinct topic — e.g. `research-auth-providers.md`, `research-perf-benchmarks.md` | `reference` |
| `decisions.md`        | ADRs, options, rationale                     | Any non-obvious tradeoff. For large efforts, split per decision: `decision-database-choice.md`                      | `note`      |
| `scratch-<label>.md`  | Drafts, working notes, exploratory code      | Complex reasoning or prototyping. Disposable — can be deleted after use                                             | `snippet`   |
| `<deliverable>.md`    | Final outputs                                | Reports, summaries, documentation                                                                                   | `artifact`  |
| `references.md`       | Links to external resources, docs, prior art | When multiple sources inform the work                                                                               | `reference` |

Always pass `--desc "why this file exists"` — it's the most valuable metadata in the viewer.

### Splitting heuristic

- If a section in any file exceeds ~80 lines, extract it into its own file
- If you're about to add a second `## Research:` or `## Decision:` heading to an existing file, create a new file instead
- `plan.md` should stay lean — it's the map, not the territory. Link to detail files:
  ```markdown
  ## Research
  - [Auth providers](research-auth-providers.md)
  - [Performance](research-perf-benchmarks.md)
  ```

### plan.md Template

```markdown
# Plan: [Brief Description]

## Goal
[One sentence describing the end state]

## Phases
- [ ] Phase 1: [Description]
- [ ] Phase 2: [Description]
- [ ] Phase 3: [Description]

## Status
**Current:** [What's happening now]

## Decisions
- [Decision]: [Rationale]

## Errors Encountered
- [Error]: [Resolution]
```

### research.md Template

```markdown
# Research: [Topic]

## Sources
- [Source]: [Key findings]

## Findings
### [Category]
- [Finding]
```

### decisions.md Template

```markdown
# Decisions: [Task]

## [Decision Title]
**Status:** Decided | Pending
**Options:**
1. [Option A] - [Pros/Cons]
2. [Option B] - [Pros/Cons]

**Choice:** [Selected option]
**Rationale:** [Why]
```

## Task System Integration

Planning files and tasks serve different stages:

| Stage             | Tool                | Purpose                                    |
| ----------------- | ------------------- | ------------------------------------------ |
| **Investigation** | Planning Files      | Research, decisions, rationale, error logs |
| **Execution**     | TaskCreate/TaskList | Decomposed work items with dependencies    |

### Workflow

```
1. scratch new "<task>" --dir _plans --id "$SESSION_ID"   # create the pad + plan.md
2. Research and document in planning files; `scratch add` each one
3. Make decisions, document rationale
4. Outline ALL tasks in plan.md first — present to user for review
5. After user approves, batch-create all tasks via TaskCreate
6. Link tasks back to planning docs
7. scratch ui "<task>" --dir _plans   # backgrounded, for the human
```

### Outline Before Creating Tasks

Do NOT create tasks one by one as you discover them. Instead:

1. Collect all phases and tasks during investigation, write them as a checklist in `plan.md`
2. Present the full outline to the user via `AskUserQuestion` — they need to see the complete picture before committing
3. Only after approval, batch-create **all** tasks in a single pass — both phase-level and work-item tasks

This matters because the user needs to evaluate scope, reorder priorities, and spot gaps — which is impossible when tasks trickle in one at a time.

### Task Hierarchy

Create tasks at two levels:

1. **Phase tasks** — one per phase, owns the high-level goal. Mark complete when all child tasks are done.
2. **Work-item tasks** — concrete implementation steps within a phase.

Example batch after outline approval:
```
TaskCreate: subject: "Phase 1: Research auth providers"
TaskCreate: subject: "Phase 1.1: Compare OAuth2 libraries"
TaskCreate: subject: "Phase 1.2: Evaluate token storage options"
TaskCreate: subject: "Phase 2: Implement auth flow"
TaskCreate: subject: "Phase 2.1: Add OAuth2 login endpoint"
TaskCreate: subject: "Phase 2.2: Add token refresh middleware"
```

Phase tasks give the user a progress overview; work-item tasks track actual execution.

### Linking Tasks to Planning Documents

When batch-creating, reference the pad dir in each task:

```
TaskCreate:
  subject: "Add OAuth2 login endpoint"
  description: |
    Implement /auth/login endpoint.
    References: _plans/2026-01-23-auth-api/plan.md
  metadata:
    planDir: "_plans/2026-01-23-auth-api"
```

Tasks can have dependencies on each other, and all link back to the same pad directory for context.

## Recommended Tools

| Tool                 | When to Use                                                                       |
| -------------------- | --------------------------------------------------------------------------------- |
| **Explore subagent** | Search codebase, find patterns, understand existing structure before planning     |
| **AskUserQuestion**  | Clarify requirements, validate assumptions, confirm decisions before proceeding   |
| **scratch ui**       | Open the pad in the visual viewer (backgrounded) so the human can browse the plan |

Use these proactively during investigation to avoid wrong assumptions and wasted effort.

## Success Criteria

Planning is complete when:
- [ ] A pad exists under `_plans/` with `plan.md` (clear goal and phases) registered
- [ ] All phases marked complete or explicitly deferred
- [ ] Key decisions documented with rationale
- [ ] Errors encountered are logged with resolutions
- [ ] All files registered with `scratch add` (visible in the viewer)
- [ ] User confirms deliverables via `AskUserQuestion`

## Critical Rules

### Refresh Goals When Context Gets Long

After many tool calls (~20+), re-read `plan.md` before major decisions. This brings goals back into the attention window.

### 1. Store, Don't Stuff
Large outputs go to files, not context. Keep paths in working memory, content in files.

### 2. Log All Errors
Every error goes in plan.md under "Errors Encountered". This builds knowledge and shows recovery.

### 3. Decisions Need Rationale
Don't just record what you decided - record WHY. Future-you needs this context.

### 4. Update Status Immediately
Mark phases complete as soon as they're done. Don't batch status updates.

## Anti-Patterns

| Don't                                            | Do Instead                                               |
| ------------------------------------------------ | -------------------------------------------------------- |
| Create files in project root                     | Use a pad under `_plans/YYYY-MM-DD-name/`                |
| Write files but forget to register them          | `scratch add` each file so it shows in the viewer        |
| State goals once and forget                      | Re-read plan.md when context is long                     |
| Hide errors and retry silently                   | Log errors with resolution                               |
| Stuff everything in context                      | Store large content in files                             |
| Start executing immediately                      | Create plan.md first for complex tasks                   |
| Put research + decisions + notes in one big file | One file per concern — split by topic                    |
| Let plan.md grow past ~80 lines                  | Extract sections into dedicated files, link from plan.md |

Also works on pi

If you’re on the pi coding agent, the @nikiforovall/pi-scratchpad package ships the same skills plus /scratch ui | export | stop commands for the viewer:

pi install npm:@nikiforovall/pi-scratchpad

Same idea, same CLI underneath — give your agent’s working memory somewhere to land.

Summary

scratch turns the throwaway-but-useful output of an agent session into durable, inspectable, shareable knowledge: a folder, a manifest, and a viewer. It’s deliberately small — the agent does the writing, the filesystem does the storing, and scratch just keeps track.

Reference

pi-otel: OpenTelemetry Tracing for the Pi Coding Agent

Sat, 16 May 2026 00:00:00 +0000

TL;DR: pi-otel is an OpenTelemetry extension for the pi coding agent. It emits one trace tree per user prompt — interaction, turns, LLM requests, and tool calls — over OTLP. The default target is a local .NET Aspire dashboard. Any OTLP-compatible backend (Grafana LGTM, Jaeger, Honeycomb) works too. Install with pi install npm:pi-otel, start with /otel start.

Docs: nikiforovall.blog/pi-otel/

Why telemetry for a coding agent?

Pi is deliberately minimal. That minimalism is a feature — but it means there’s no built-in timeline view of what the agent actually did. You can read the session log, but a log is not a trace. You can count tokens from the session info modal, but a histogram is not a counter.

Honestly, the first reason to install it is just curiosity. Watching a span tree populate in real time — seeing exactly which tool ran, how long it took, what the model returned — is a different kind of understanding than reading a session log after the fact. If you’ve ever wondered what pi is actually doing under the hood, a trace will show you in a way that no amount of log-reading will.

If you’re running pi in a CI pipeline or as part of a larger workflow, “it worked or it didn’t” is not enough. You want to know which tool call ate 40 seconds, why a particular prompt costs twice as much as others, and whether errors cluster around a specific model or tool. That’s what spans give you.

What it emits

One trace tree per user prompt:

pi.interaction                      ← root span (one per turn)
└── pi.turn                         ← one per agent turn
    ├── pi.llm_request              ← the LLM call
    └── pi.tool.<name>              ← one per tool execution

Every span carries GenAI semantic convention attributes — gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, finish reason, tool call IDs, and gen_ai.conversation.id to correlate spans across a session.

The session is intentionally not a span. A pi session can run for hours; long-running root spans are an OTel anti-pattern. Instead, session identity travels as attributes on every span.

Quickstart

pi install npm:pi-otel

Then inside pi:

/otel start

That’s it. pi-otel auto-detects an OTLP backend in this order: Aspire CLI → Docker → Podman. It spawns a local .NET Aspire dashboard and opens it at http://localhost:18888.

Backend install options:

Aspire CLI — irm https://aspire.dev/install.ps1 | iex (Windows) or curl -sSL https://aspire.dev/install.sh | bash
Docker or Podman — any recent version

What you see in Aspire

The Traces tab shows one root span per user prompt. Expand to see the full turn → LLM request → tool call nesting. Click any span for the attribute panel: model, token counts, finish reason, tool input/output.

The Metrics tab (enabled with "signals": { "metrics": true }) shows histograms for LLM request latency, token usage (input/output/cache), and tool execution time.

The Structured Logs tab (enabled with "signals": { "logs": true }) surfaces lifecycle events — pi.session.start, pi.session.end, pi.tool.error — plus the filtered OTel SDK diag bridge output.

Any OTLP backend

Aspire is the default because it requires zero configuration. But pi-otel is built on open standards — the OTLP exporter works with anything.

Grafana LGTM

The samples/lgtm/ directory ships a docker-compose.yml that spins up the otel-lgtm image (Tempo + Mimir + Loki in one container):

cd samples/lgtm
docker compose up -d

Then inside pi:

/otel connect http://localhost:4317

Traces land in Tempo, metrics in Mimir/Prometheus, logs in Loki — all pre-wired to Grafana.

Other backends

For Honeycomb, Grafana Cloud, Jaeger, or any other OTLP receiver, point the endpoint and add auth headers:

// .pi/settings.json
{
  "otel": {
    "endpoint": "https://api.honeycomb.io",
    "headers": { "x-honeycomb-team": "YOUR_API_KEY" }
  }
}

Or via env vars:

OTEL_EXPORTER_OTLP_ENDPOINT=https://api.honeycomb.io
OTEL_EXPORTER_OTLP_HEADERS=x-honeycomb-team=YOUR_API_KEY

Configuration reference

// .pi/settings.json or ~/.pi/agent/settings.json
{
  "otel": {
    "enabled": true,
    "endpoint": "http://localhost:4317",
    "protocol": "grpc",
    "serviceName": "pi",
    "captureContent": "metadata_only",
    "sampleRatio": 1.0,
    "signals": { "traces": true, "metrics": false, "logs": false }
  }
}

Key	Default	Description
`enabled`	`true`	Master switch
`endpoint`	`http://localhost:4317`	OTLP receiver URL
`protocol`	`grpc`	`grpc`, `http/protobuf`, or `http/json`
`captureContent`	`metadata_only`	Content capture mode
`sampleRatio`	`1.0`	Head sampling ratio (0.0–1.0)
`signals.traces`	`true`	Emit trace spans
`signals.metrics`	`false`	Emit token/cost/latency histograms
`signals.logs`	`false`	Emit lifecycle log records + diag bridge

Standard OTEL_* env vars override settings. PI_OTEL_DISABLED=1 makes the extension a complete no-op.

Content capture

By default (captureContent: "metadata_only") spans carry token counts, model names, finish reasons, and tool call IDs — but not prompt or response text. Three modes:

Mode	What lands on spans
`metadata_only`	Token counts, model, finish reason, tool IDs
`no_tool_content`	+ LLM message content; no tool input/output
`full`	Everything, including tool input/output (capped at 60 KB per attribute)

{ "otel": { "captureContent": "full" } }

Extensibility — logs from your own pi package

Since pi-otel registers a global OTel LoggerProvider at startup, any code in the same process can emit log records into the same stream. Two ways to do it:

OTel API directly — add @opentelemetry/api-logs as a dependency, call the global provider. The call is a no-op if pi-otel isn’t loaded.

import { logs, SeverityNumber } from "@opentelemetry/api-logs";

logs.getLogger("my-package", "1.0.0").emit({
  severityNumber: SeverityNumber.DEBUG,
  body: "cache hit",
  attributes: { "tool.name": "bash" },
});

pi-otel:log event bus — no import needed. From any pi extension, emit on pi’s shared event bus.

pi.events.emit("pi-otel:log", {
  eventName: "my-package.cache-hit",
  severity: "debug",
  body: "cache hit",
  attributes: { "tool.name": "bash" },
});

Both land in Aspire Structured Logs alongside pi-otel’s own session and error events. The OTel API approach uses your own instrumentation scope; the event bus emits under pi-otel.

The diagnostic bridge

When signals.logs is enabled, pi-otel forwards the @opentelemetry/* SDK’s own internal diag messages to the same OTLP endpoint under the @opentelemetry/diag instrumentation scope.

Getting started

pi install npm:pi-otel

Then in pi:

/otel start         # spawn local Aspire dashboard

Full documentation: nikiforovall.blog/pi-otel/

Summary

Pi’s minimalism is a strength. pi-otel adds one thing: a timeline view of what the agent actually did, in a format that integrates with your existing observability stack. One trace tree per turn, three signals, any OTLP backend.

🙌 I hope you found it helpful. If you have any questions, please feel free to reach out. If you’d like to support my work, a star on GitHub would be greatly appreciated! 🙏

References

pi-kanban: A Workspace for the Pi Coding Agent

Sat, 09 May 2026 00:00:00 +0000

TL;DR: pi-kanban is a read-only observability dashboard for the pi coding agent. It watches ~/.pi/agent/sessions and renders sessions, todos, messages, and subagents in real time. Install with pi install npm:pi-kanban, then run /kanban start.

Source code: github.com/NikiforovAll/pi-kanban

Why agent observability matters

Working with a coding agent is not the same as writing code yourself. The agent makes decisions, calls tools, spawns subagents, and produces artifacts faster than you can read. Without a way to see what’s happening, you end up babysitting a terminal or trusting the output blindly.

A proper observability surface changes a few things:

Reduced cognitive load. You glance, not read. Status, progress, and what the agent is doing right now are visible without scrolling logs.
Better alignment. Plan, todos, and messages sit side by side, so course-correction is cheap. You catch drift early instead of after the fact.
A workspace, not a viewer. Pinned sessions, pinned messages, linked docs, markdown previews. The work lives in the dashboard, not just a window into it.

That last one is the part I care about most. pi-kanban is read-only against pi’s session state, but the workspace around it is not.

What it looks like

Three panes:

Left — sessions grouped by project, with progress, age, and pin state.
Center — kanban board (Pending / In Progress / Completed) for the selected session.
Right — Session Log: messages, tool calls, subagent activity.

A bottom strip shows recent subagent runs across all sessions.

Feature coverage

Sessions and projects

pi-kanban reads ~/.pi/agent/sessions/<encoded-cwd>/<timestamp>_<id>.jsonl and groups sessions by working directory. No daemon, no database — just file watching.

Pinning sessions

Long-running work doesn’t fit one session. Pinning lifts a session out of the auto-sort so it stays at the top. It’s how you keep the thing you’re actually working on in front of you while exploratory sessions come and go.

/kanban pin <session-id>
/kanban sticky-pin <session-id>

Pinning messages and linking documents

Inside a session, you can pin individual messages. Useful for the assistant message that contains the answer, or the user message that defined the task. Linked documents and the scratchpad let you attach context that isn’t part of the conversation but should travel with it.

This is what turns the dashboard into a workspace. The things you care about stay anchored even as the session log scrolls past them.

Subagents

When pi-subagents spawns a child session, pi-kanban nests it under the parent.

Session info

The info modal shows model, token usage, cache hit rate, cost, duration, and paths. The things you check before deciding whether a session is worth resuming.

Storage manager

Lists sessions, scratchpads, and linked docs with size accounting. Used to clean orphaned docs and unlink stale references.

Follow last message

Pop the latest assistant message into a floating window.

Themes

Four built-in themes (pi-light, pi-dark, kanban-default, kanban-dark-default) and user-defined themes via ~/.pi/agent/kanban/settings.json.

Slash commands

Command	What it does
`/kanban start`	Start the local server (port 3460) in the background
`/kanban stop`	Stop the running server
`/kanban open`	Open the dashboard in the default browser
`/kanban app`	Open in a standalone PWA window (if installed)
`/kanban pin <id>`	Pin a session to the top of the sidebar
`/kanban sticky-pin <id>`	Pin and keep across restarts
`/kanban preview <path>`	Render a markdown file in the dashboard preview pane
`/kanban link <id>`	Links files to a session

Getting started

pi install npm:pi-kanban
# inside pi:
/kanban start
/kanban open

Here is a live demo of pi-kanban running on synthetic data:

Conclusion

If you use pi and want a dashboard that turns sessions into a workspace, give pi-kanban a try. Questions and feedback welcome on GitHub.

🙌 I hope you found it helpful. If you have any questions, please feel free to reach out. If you’d like to support my work, a star on GitHub would be greatly appreciated! 🙏

References

Hangfire as an MCP Operations Pane for AI Agents

Sat, 02 May 2026 00:00:00 +0000

TL;DR

Nall.Hangfire.Mcp is an in-process library. It exposes your Hangfire jobs at /mcp as MCP tools, and ships a built-in maintenance toolset and a few prompts on the side. Call AddHangfireMcp() and MapHangfireMcp("/mcp") inside the same ASP.NET Core host that already runs Hangfire, and the schemas, descriptions, auth hook, and maintenance tools come with it.

📖 Documentation: https://nikiforovall.github.io/hangfire-mcp-dotnet/

Package	Version	Source	Description
`Nall.Hangfire.Mcp`		hangfire-mcp-dotnet	In-process MCP server for Hangfire jobs and queue maintenance.
`Nall.Hangfire.Mcp.Generator`		hangfire-mcp-dotnet	Roslyn source generator for compile-time job discovery.

1. Why an Operations Pane

Hangfire’s dashboard is a fine UI for humans. It’s a poor surface for agents. Two specific things are missing.

1️⃣ The first is parameterized job invocation. The dashboard is built around retrying and inspecting jobs that already exist. It doesn’t give you a clean way to say “run a job for me with parameters” from chat or a copilot.

2️⃣ The second is operational workflows. “Show me the current backlog.” “List failures from the last hour grouped by exception type.” “Requeue the timeouts, but only the ones older than five minutes.” Most teams end up writing these as one-off scripts.

MCP is a good fit for both. It standardizes how a client discovers tools, inspects schemas, and invokes them. Nall.Hangfire.Mcp puts two kinds of tools on /mcp: job tools, one Run_<jobId> per discovered Hangfire job; and maintenance tools (statistics, queue listing, get/delete/requeue with dryRun), plus three orchestration prompts (hangfire_health_check, hangfire_triage_failures, hangfire_discover).

💡 That mix is what I’m calling an Operations Pane. Hangfire is the system of record. MCP is the discovery and invocation channel. The operator on the other end is whatever you want it to be: Claude, Copilot, MCP Inspector, your own client.

flowchart LR agent["AI Agent"] -->|MCP| mcp[/"/mcp endpoint"/] mcp --> dispatch{Tool kind?} dispatch -->|Run_jobId| sched[HangfireDynamicScheduler] dispatch -->|hangfire_*| maint[MaintenanceDispatcher] sched --> client[IBackgroundJobClient] maint --> storage[(Hangfire JobStorage)] client --> storage storage --> worker[Hangfire Server
executes job]

Aspire bonus 🎁

Aspire 13.x knows about MCP, so the AppHost wiring is short. .WithMcpServer("/mcp") declares an MCP endpoint on a project resource, and AddMcpInspector(...) from CommunityToolkit.Aspire.Hosting.McpInspector deploys the official MCP Inspector as a companion service pointed at your server.

var web = builder.AddProject<Projects.Web>("server").WithMcpServer("/mcp");

builder.AddMcpInspector("inspector").WithMcpServer(web);

Two lines of AppHost code, and the Inspector boots up alongside your app pointed at /mcp. External clients (Claude Desktop, VS Code Copilot, an SDK) get the same endpoint without any tool authoring on your side.

2. Getting started

dotnet add package Nall.Hangfire.Mcp

The minimal wire-up on top of an existing Hangfire host is two lines:

builder.Services.AddHangfireMcp();
app.MapHangfireMcp("/mcp");

Every recurring job in Hangfire’s storage now shows up as a Run_<jobId> MCP tool, and the hangfire_* maintenance tools and prompts are wired up. To pick up one-shot enqueue sites too, opt into the source generator with o.Sources = JobDiscoverySources.All.

2.1 How a job gets discovered

There are two discovery sources, controlled by HangfireMcpOptions.Sources. RecurringStorage (the default) reads RecurringJobDto.Job entries from JobStorage at startup. StaticManifest runs at compile time: a Roslyn source generator walks AddOrUpdate / Enqueue / Schedule call sites and emits a manifest the runtime consumes — that’s how one-shot enqueues that never live in recurring storage end up in the catalog.

Both feed a single JobCatalog, deduped by (DeclaringType, MethodInfo). Schemas are generated once per method and respect both C# defaults and nullable annotations.

flowchart LR subgraph compile [Compile time] src[AddOrUpdate / Enqueue /
Schedule call sites] -->|Roslyn| gen[Source generator] gen --> manifest[Static manifest] end subgraph runtime [Runtime] rec[(JobStorage
recurring)] --> catalog manifest --> catalog[JobCatalog] catalog -->|reflection| schema[JSON Schema
per method] schema --> tools["MCP tools
Run_jobId"] end

2.2 Decorating jobs with `[Description]`

Tool selection in agents lives or dies by description quality. Generic auto-generated descriptions and an agent will pick the wrong tool or fill arguments badly. Nall.Hangfire.Mcp honors the standard System.ComponentModel.DescriptionAttribute at both the method and parameter level — no custom attribute types.

public interface IReportJob
{
    [Description("Generate the annual financial report and persist it.")]
    Task GenerateAsync(
        [Description("Calendar year of the report (e.g. 2026).")] int year,
        [Description("Output format. Supported: pdf, html, csv.")] string format = "pdf",
        [Description("Optional cutoff timestamp.")] DateTimeOffset? since = null);
}

The method [Description] becomes the tool’s description. Each parameter [Description] becomes the JSON Schema description for that property. format is optional because of the C# default; since is optional because of the nullable annotation. Only year ends up in required.

3. Demo: MCP Inspector

End-to-end, with the Aspire AppHost running, the loop is: list the tools on /mcp, fill in parameters in MCP Inspector, hit Run Tool, and see the same enqueue land in the Hangfire dashboard.

1. Invoking with parameters — JSON Schema rendered as a form.

2. Result in the Inspector — Hangfire job ID and state.

3. Same job in the Hangfire dashboard — exact args, real queue.

The point: the agent didn’t need to know anything about Hangfire. It saw a tool, filled in the schema, and the queue did its usual thing.

4. Authentication and authorization

The library is auth-agnostic on purpose. MapHangfireMcp("/mcp") returns IEndpointConventionBuilder, so you layer ASP.NET Core auth on top with .RequireAuthorization(...) like any other endpoint.

The full picture for a tool call looks like this:

sequenceDiagram autonumber participant Agent as AI Agent participant MCP as /mcp participant Auth as ASP.NET Core auth participant Pipe as JobInvocationPipeline participant HF as Hangfire Agent->>MCP: CallTool (no token) MCP-->>Agent: 401 + WWW-Authenticate (RFC 9728) Agent->>Agent: discover OAuth metadata, obtain JWT Agent->>MCP: CallTool (Bearer JWT) MCP->>Auth: validate token, RequireAuthorization Auth->>Pipe: invoke tool Pipe->>Pipe: evaluate [Authorize] on job alt allowed Pipe->>HF: Enqueue job HF-->>Agent: tool result else denied Pipe-->>Agent: MCP error (Forbidden) end

Authentication

The sample uses Keycloak as the OAuth 2.0 / OIDC authorization server, JWT bearer for token validation, and the MCP-spec McpAuthenticationHandler to emit RFC 9728 challenges. That last bit matters: it lets MCP clients self-discover the protected-resource metadata at /.well-known/oauth-protected-resource/mcp, which is how you avoid hardcoding auth server URLs into clients.

app.MapHangfireMcp("/mcp")
    .RequireAuthorization(p => p
        .RequireAuthenticatedUser()
        .AddAuthenticationSchemes(McpAuthenticationDefaults.AuthenticationScheme));

The full setup (Keycloak realm import, JWT + AddMcp(...) resource metadata, the container-vs-host issuer URL caveat that bit me twice) is in the repo at docs/authentication.md. Other ASP.NET Core schemes work the same way: Entra ID, Auth0, custom JWT — nothing about the library cares which one you pick.

Per-job authorization

Endpoint auth is necessary but not enough. Once a client is past the /mcp gate it can list every tool, and you usually want different jobs guarded by different policies. So Nall.Hangfire.Mcp honors the standard [Authorize] attribute on job methods — opt in with AddJobAuthorization(), then decorate:

[Authorize(Policy = "jobs:run")]
Task GenerateAsync(int year, string format = "pdf", DateTimeOffset? since = null);

Under the hood, AddJobAuthorization() registers an AuthorizeAttributeFilter into the job invocation pipeline. When a tool call hits HangfireMcpHandlers, the pipeline collects [Authorize] attributes from the method and its declaring type, evaluates them against the current ClaimsPrincipal via IAuthorizationService, and short-circuits to an MCP error if any check fails. No Hangfire shim, no custom attribute. It’s the same [Authorize] you’d put on a controller action.

That keeps the threat model boring, which is what you want. The /mcp endpoint is gated by RequireAuthorization(). Per-job access is gated by ordinary ASP.NET Core policies. Catalog visibility and execution rights are separate concerns: an agent that lacks the right claims still sees jobs in the catalog it can’t run, and the call returns Forbidden when it tries.

Summary

Nall.Hangfire.Mcp collapses the “expose Hangfire to an MCP client” problem into a couple of lines of host code. Discovery comes from two sources (recurring storage at runtime, a Roslyn source generator at compile time). Schemas come from method signatures, with [Description] doing the heavy lifting. Auth and authorization reuse the standard ASP.NET Core stack: RequireAuthorization() on the endpoint, [Authorize(Policy = ...)] on the job.

Inside an Aspire AppHost, .WithMcpServer plus AddMcpInspector is most of the integration work. What you end up with is what I keep calling an Operations Pane: one surface where agents can run jobs and operate the queue, and where the security model is the same one you already use for the rest of the app.

Reference

User-Managed Access (UMA) 2.0 Resource Sharing with Keycloak and .NET

Wed, 15 Apr 2026 00:00:00 +0000

TL;DR

UMA 2.0 lets resource owners control access to their stuff. Think Google Drive sharing, but wired into your authorization server.
Keycloak.AuthServices now supports the challenge-response flow (automatic ticket-for-RPT exchange) and async approval (request, wait for owner, get in).
A Blazor Server + Minimal API sample shows the full flow. UmaTokenHandler handles the UMA dance behind the scenes.
IKeycloakProtectionClient covers the permission ticket lifecycle: create, list, approve, deny.

Source code: https://github.com/NikiforovAll/keycloak-authorization-services-dotnet/tree/main/samples/UmaResourceSharing

What is UMA?

Most authorization systems work the same way: the resource server decides who gets access based on roles, claims, or policies that an admin defined. But what if the resource owner, not the admin, should make that call?

User-Managed Access (UMA) is an OAuth 2.0 extension that flips this. Alice can grant or revoke access to her resources for bob, without an admin involved, and without needing to be online when bob asks.

You’ve seen this pattern before:

Sharing a Google Doc with specific people
Granting a colleague access to a private GitHub repo
Approving a permission request in a cloud console

What makes UMA different from regular OAuth is the asynchronous approval step. Bob requests access, alice reviews it later, and only then does bob get in.

Key concepts

Concept	Description
Resource Owner	The user who owns the protected resource
Requesting Party	A user who wants access to someone else's resource
Permission Ticket	A one-time challenge token representing an access request
RPT (Requesting Party Token)	An access token enriched with specific resource permissions
Protection API	Keycloak API for managing resources, permissions, and tickets

Demo

Here’s what this looks like in the Blazor sample.

alice (resource owner), instant access

Alice owns shared-document, so authorization succeeds right away:

bob requests access, alice approves

Bob has no permission. He submits a request, alice approves it, then bob retries and gets in:

bob (no permission), access denied

Without approval, bob gets a clear “Access Denied”:

How the UMA flow works

Two flows, working together.

Challenge-response (instant access)

When a user already has permission, the client handles the exchange automatically:

sequenceDiagram participant User as User (Browser) participant Client as Client App participant RS as Resource Server participant KC as Keycloak User->>Client: Access protected resource Client->>RS: GET /resource (Bearer token) RS->>KC: Evaluate permissions KC-->>RS: 403 (no permission yet) RS->>KC: Create permission ticket KC-->>RS: ticket="abc123..." RS-->>Client: 401 WWW-Authenticate: UMA ticket="abc123..." Client->>KC: Exchange ticket for RPT KC-->>Client: RPT (with permissions) Client->>RS: GET /resource (Bearer RPT) RS-->>Client: 200 OK — resource content

UmaTokenHandler handles steps 6-9. Your code just makes HTTP calls as usual.

Async approval (request, approve, access)

When a user has no permission, they submit a request. The resource owner reviews it later:

sequenceDiagram participant Bob as Bob participant App as Application participant KC as Keycloak participant Alice as Alice (Owner) Bob->>App: Request access to resource App->>KC: Create permission ticket (granted=false) KC-->>App: 201 Created App-->>Bob: "Request submitted" Note over Alice: Alice logs in later Alice->>App: View pending requests App->>KC: GET permission tickets (granted=false) KC-->>App: Pending tickets list Alice->>App: Approve bob's request App->>KC: Update ticket (granted=true) Note over Bob: Bob retries Bob->>App: Access resource App->>KC: Evaluate permissions KC-->>App: Permitted App-->>Bob: Resource content

Architecture

Two projects: a Blazor Server client talks to a Minimal API resource server. Keycloak sits in the middle, handling authorization decisions and permission tickets.

Resource server

The resource server protects endpoints with RequireProtectedResource() and returns WWW-Authenticate: UMA challenges on authorization failure via AddUmaPermissionTicketChallenge():

services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddKeycloakWebApi(configuration, options =>
    {
        options.Audience = "uma-resource-server";
        options.RequireHttpsMetadata = false;
    });

// Authorization + UMA challenge handler
services.AddAuthorization()
    .AddKeycloakAuthorization()
    .AddUmaPermissionTicketChallenge();

services.AddAuthorizationServer(configuration)
    .AddStandardResilienceHandler();

// Protection API for ticket management
services.AddKeycloakProtectionHttpClient(configuration)
    .AddClientCredentialsTokenHandler(tokenClientName);

// Protected endpoints
app.MapGet("/documents/{name}", (string name) =>
        new DocumentResponse(name, $"Content of {name}"))
    .RequireProtectedResource("shared-document", "read");

app.MapGet("/documents/{name}/details", (string name) =>
        new DocumentResponse(name, $"Detailed content of {name}"))
    .RequireProtectedResource("shared-document", "write");

It also exposes permission ticket management endpoints:

// Bob submits an access request
app.MapPost("/permissions/request", async (...) =>
{
    // ...
    var ticket = new PermissionTicket
    {
        Resource = resourceId, Requester = userId,
        ScopeName = "read", Granted = false,
    };
    await protectionClient.CreatePermissionTicketWithResponseAsync(realm, ticket);
}).RequireAuthorization();

// Alice lists pending requests
app.MapGet("/permissions/pending", async (...) =>
{
    return await protectionClient.GetPermissionTicketsAsync(realm,
        new GetPermissionTicketsRequestParameters { Granted = false, ReturnNames = true });
}).RequireAuthorization();

// Alice approves
app.MapPut("/permissions/{id}/approve", async (...) =>
{
    await protectionClient.UpdatePermissionTicketAsync(realm,
        new PermissionTicket { Id = id, Granted = true });
}).RequireAuthorization();

Client app (Blazor Server)

The Blazor client registers UmaTokenHandler from Keycloak.AuthServices.Authorization.Uma. It’s a DelegatingHandler that intercepts 401+UMA responses and handles the ticket exchange:

// UMA ticket exchange client
services.AddKeycloakUmaTicketExchangeHttpClient(configuration);

// HTTP client with UMA handling
services.AddHttpClient("ResourceServer", (sp, client) =>
    {
        var config = sp.GetRequiredService<IConfiguration>();
        var baseUrl = config["services:resource-server:http:0"];
        client.BaseAddress = new Uri(baseUrl);
    })
    .AddUmaTokenHandler();

What UmaTokenHandler does:

Attaches the user’s access token to outgoing requests
On 401 with WWW-Authenticate: UMA, extracts the permission ticket
Exchanges the ticket for an RPT at Keycloak’s token endpoint
Retries the original request with the RPT

From the Blazor component side, none of this is visible. You just make HTTP calls:

var response = await Http.GetAsync($"/documents/{documentName}");
if (response.IsSuccessStatusCode)
{
    var document = await response.Content.ReadFromJsonAsync<Document>();
}

Keycloak setup

UMA needs a few things configured in Keycloak:

A confidential client with Authorization Services enabled (the resource server)
A resource with ownerManagedAccess: true and scopes like read, write
An owner policy that grants the resource owner access
An OIDC client for user login, with an audience mapper pointing to the resource server

The sample ships with pre-configured Keycloak realm exports, so you can just run it:

dotnet run --project samples/UmaResourceSharing/AppHost

Aspire opens the dashboard automatically with Keycloak, the resource server, and the Blazor app.

Summary

Role-based access control doesn’t cover the case where users share resources with each other. UMA does.

With Keycloak.AuthServices you get UmaTokenHandler for the challenge-response dance, AddUmaPermissionTicketChallenge() for resource servers, and IKeycloakProtectionClient for managing permission tickets. The sample runs with a single dotnet run via Aspire.

References

Join us on Discord:

Claude Code Hub: Kanban, Marketplace, Cost, and Memory in One Place

Wed, 08 Apr 2026 00:00:00 +0000

Too many tabs

I built a bunch of small tools around Claude Code over the past few months. A Kanban board for watching agent work. A marketplace for managing plugins. A cost dashboard. A memory explorer. Each one is its own npx command and its own browser tab, and at some point I got tired of juggling four terminals and four tabs every time I sat down to work.

So I wrote a hub that runs them all behind one command.

npx claude-code-hub --open

One process spawns four servers, opens a browser, done. The tools live in iframes and you switch between them with keyboard shortcuts. No visible chrome, no tabs.

What’s inside

Kanban

A real-time task board for Claude Code sessions. It shows tasks, agents, and subagent teams as Claude works. State comes from JSONL files written by lightweight hooks (more on that below).

Marketplace

Browse and install Claude Code plugins across local, user, and project scopes. Basically an app store for skills and MCP servers.

Cost

Token usage and cost breakdowns. Drill from totals down to projects, then to individual sessions. I check this more often than I’d like to admit.

Memory

Shows all memory sources that feed into Claude Code: CLAUDE.md files, rules, auto memory, import chains. Really helpful when Claude does something unexpected and you want to figure out which instruction is responsible.

How it actually works

The hub server spawns each sub-app as a child process on its own port. If a port is busy, the tool picks a random free one. The hub reads stdout to detect the actual port each tool landed on, then serves a shell page that embeds them as iframes.

Switching is keyboard-only:

Shortcut	Action
`Alt+1`	Kanban
`Alt+2`	Marketplace
`Alt+3`	Cost
`Alt+4`	Memory
`Ctrl+Alt+Arrow`	Next / previous tool

Tools can also deep-link to each other via postMessage. Click a session in Cost and it can jump you straight to that session’s Kanban board.

You can install it as a PWA for a standalone window with no browser UI. That’s the way I use it, honestly. It feels like its own app.

The whole thing is vanilla JS and Express. No bundler, no build step, works on Node 18+.

Setup

1. Install hooks (once)

For the full Kanban experience, you need to install hooks that log agent activity:

npx claude-code-kanban --install

These are small shell scripts that append to JSONL files. Negligible overhead. Without them you still get the task board, just no live agent indicators.

2. Launch

npx claude-code-hub --open

3. Use Claude Code normally

The hub updates live. Switch tools with the keyboard while Claude works.

Standalone mode

Each tool also works on its own if you only want one:

npx claude-code-kanban          # task board
npx claude-code-marketplace     # plugin manager
npx claude-code-cost            # cost dashboard
npx claude-code-memory-explorer # memory explorer

Claude Code doesn’t need to be running either. You can browse past sessions and costs anytime; live updates just appear when Claude starts.

What's new in Keycloak.AuthServices v2.9.0

Fri, 27 Mar 2026 00:00:00 +0000

What’s new

Keycloak.AuthServices just got a round of updates. Keycloak itself has changed a lot between versions 24 and 26 – lightweight access tokens, organizations, RFC 8414 metadata – and the .NET library needed to catch up.

This post walks through the three biggest additions.

Package	Version	Description
`Keycloak.AuthServices.Authentication`		Authentication for API and Web Apps
`Keycloak.AuthServices.Authorization`		Authorization Services + Authorization Server integration
`Keycloak.AuthServices.Authorization.TokenIntrospection`		Token introspection for lightweight access tokens
`Keycloak.AuthServices.Sdk`		Admin API and Protection API
`Keycloak.AuthServices.Sdk.Kiota`		Admin API based on OpenAPI (Kiota-generated)

Organization authorization

Keycloak 24 introduced organizations as a first-class concept – and for multi-tenant applications, this changes things.

Before organizations, multi-tenancy in Keycloak usually meant one realm per tenant (operational headache) or custom attributes and groups hacked together to represent “who belongs where”. Organizations give you a built-in abstraction: users belong to organizations, organizations have their own identity providers, and membership shows up directly in the token.

For a SaaS app, this means you can stop reinventing tenant isolation. Keycloak manages the “which user belongs to which tenant” question, and the token carries that answer. Your API just needs to enforce it.

The library picks that up and gives you authorization policies that work with it.

Keycloak produces two claim formats depending on the Organization Membership Mapper configuration – a simple string array or a rich JSON map with IDs and attributes. Both are handled transparently.

To include organization membership in tokens, request the organization:* scope. The plain organization scope alone does not include the claim.

Require any organization membership

The simplest check – the user must belong to at least one organization:

app.MapGet("/orgs", () => "You belong to an org")
    .RequireOrganizationMembership();

Require a specific organization

app.MapGet("/acme/settings", () => "Acme settings")
    .RequireOrganizationMembership("acme-corp");

Resolve the organization from route or header

Sometimes the organization comes from the request itself. The library ships with built-in resolvers for routes, headers, and query parameters:

app.MapGet("/orgs/{orgId}/projects", (string orgId) => $"Projects for {orgId}")
    .RequireOrganizationMembership("{orgId}");

app.MapGet("/tenant/projects", () => "Tenant projects")
    .RequireOrganizationMembership<RouteHandlerBuilder, HeaderParameterResolver>("{X-Organization}");

Policy-based approach

If you prefer named policies:

services.AddKeycloakAuthorization()
    .AddAuthorizationBuilder()
    .AddPolicy("AcmeOnly", policy =>
        policy.RequireOrganizationMembership("acme-corp")
              .RequireRealmRoles("user"));

Imperative authorization

For more dynamic scenarios:

var result = await authorizationService.AuthorizeAsync(
    user, null, new OrganizationRequirement(orgId));

The OrganizationRequirementHandler reads the organization claim, parses the JSON structure Keycloak emits, and checks membership. You can configure which claim type to look at if your setup uses a non-default claim name.

Token introspection for lightweight access tokens

This one bit me in production. Keycloak 24+ supports lightweight access tokens – valid signed JWTs, but intentionally stripped of business claims like resource_access, realm_access, and preferred_username. The idea is to keep tokens small.

The problem: role-based authorization silently fails. KeycloakRolesClaimsTransformation looks for resource_access, finds nothing, maps no roles. Every role check returns 403. No errors, no warnings. Just 403s.

The fix is token introspection. You send the token back to Keycloak’s introspection endpoint and get the full claim set.

Installation

Token introspection ships as a separate package:

dotnet add package Keycloak.AuthServices.Authorization.TokenIntrospection

Enabling introspection

services.AddKeycloakWebApiAuthentication(configuration);

services.AddKeycloakAuthorization(options =>
{
    options.EnableRolesMapping = RolesClaimTransformationSource.All;
});

services.AddKeycloakTokenIntrospection(configuration);

That’s it. The AddKeycloakTokenIntrospection call registers a claims transformation that:

Checks if expected claims (resource_access, realm_access) are already present
If missing, calls the introspection endpoint with client credentials
Merges the the full claim set into the ClaimsPrincipal
Caches results per token to avoid redundant calls

The existing role mapping transformation runs after introspection, so everything works transparently.

Configuration

Via appsettings.json:

{
  "Keycloak": {
    "realm": "my-realm",
    "auth-server-url": "https://keycloak.example.com/",
    "resource": "my-api",
    "credentials": {
      "secret": "my-client-secret"
    }
  }
}

The introspection client reuses KeycloakInstallationOptions, so if you already have Keycloak configured, you likely don’t need to add anything.

Cache duration defaults to 60 seconds and is configurable:

services.AddKeycloakTokenIntrospection(options =>
{
    configuration.BindKeycloakOptions(options);
    options.CacheDuration = TimeSpan.FromSeconds(120);
});

When to use this

You need this if:

Your Keycloak admin enabled lightweight access tokens
You’re using KC 26+ admin clients (they use lightweight tokens by default)
You see unexplained 403s after upgrading Keycloak

The alternative is configuring Keycloak protocol mappers with the “Add to lightweight access token” flag, which avoids the introspection round-trip entirely. Pick your trade-off.

RFC 8414 metadata discovery

Since Keycloak 26.4, the RFC 8414 metadata endpoint is available at /.well-known/oauth-authorization-server, alongside the traditional OIDC discovery at /.well-known/openid-configuration.

Why does this matter? If you’re building a pure OAuth 2.0 resource server – no ID tokens, no OIDC – then OIDC discovery is technically the wrong endpoint. More practically, Keycloak 26.4 added MCP (Model Context Protocol) authorization server support, which uses RFC 8414 discovery.

Using RFC 8414 metadata

services.AddKeycloakWebApiAuthentication(options =>
{
    configuration.BindKeycloakOptions(options);
    options.MetadataAddress = KeycloakConstants.OAuthAuthorizationServerMetadataPath;
});

The MetadataAddress property overrides the default OIDC discovery path. The constant OAuthAuthorizationServerMetadataPath resolves to ".well-known/oauth-authorization-server".

If you don’t set it, nothing changes – the library defaults to OIDC discovery as before.

When to use this

Machine-to-machine flows (client credentials, no ID tokens)
OAuth 2.0-only clients that expect RFC 8414 metadata
MCP authorization server scenarios with Keycloak 26.4+
Protocol correctness for pure resource servers

Also in this release

A few more things that shipped in this release:

Extensible policy builder – IProtectedResourcePolicyBuilder lets you customize how protected resource policies are constructed
Pluggable parameter resolution – custom IParameterResolver implementations for resolving resource names from routes, headers, query strings, or your own sources
Configurable organization claim type – if your Keycloak setup uses a non-standard claim name for organization data
Kiota SDK updated to Keycloak 26.5.6 OpenAPI spec

Full changelog: GitHub Releases

Documentation: nikiforovall.github.io/keycloak-authorization-services-dotnet

AI skills for Claude Code

This release also ships a Claude Code plugin with two AI skills that help you work with Keycloak directly from your terminal.

keycloak-auth-services

An implementation guide that knows the library’s API surface – authentication setup, authorization patterns, resource protection, Admin SDK, Protection API, and configuration options. Ask it how to set up JWT Bearer auth or configure protected resources and it’ll give you working code using the current API.

keycloak-administration

A Keycloak server administration guide covering realm management, client configuration, authentication flows, RBAC, user federation, security hardening, and troubleshooting. Useful when you need to configure the Keycloak side of things – setting up clients, mappers, organizations, or debugging token issues.

Join us on Discord: