Claude Code beyond prompting

Unknown — Wed, 31 Dec 2025 00:00:00 +0000

In my first week writing documentation for Claude Code, I realized I’d been using a Swiss Army knife with only the main blade, unaware of what else it could do.

Before I joined Anthropic, I thought I was pretty good at Claude Code. I was using it to speed up my technical writing: generating diagrams I couldn’t make myself, catching inconsistencies quickly, and automating the most tedious parts of doc review. Putting our entire style guide into a CLAUDE.md</code> made it even better. It wasn’t perfect, but it removed whole classes of style-related mistakes and shifted every edit closer to “done.”

That felt like a complete workflow. I’d heard about things like subagents, skills, and hooks, but what I had was working well, so I never looked much deeper.

Then I started working on the docs for these features and discovered the other tools had been there the whole time, each better suited for certain kinds of work. I just hadn’t learned when to reach for them.

Note: This is a personal mental model I’ve been building through hands-on use, not an official or exhaustive guide.

Where I started (and where you might be too)</h2>
The CLAUDE.md</code> file was my first big unlock. If you haven’t done this yet, start here: create a CLAUDE.md</code> file in your project root (or in a .claude/</code> directory) and put your universal writing guidelines there. It’s the highest-impact, lowest-effort improvement you can make.</p>
At first, I put everything in it. This included documentation philosophy, voice and tone, grammar rules, formatting conventions, and notes about our publishing workflow. Because Claude reads this file automatically at the start of every conversation, every edit followed our conventions without repeated reminders.</p>
That worked well. But as I iterated, I started adding highly specific, task-level instructions that were important but not relevant to every session. Since CLAUDE.md</code> loads in full at the start of every conversation, every extra byte counts toward your context window and token usage. It’s best suited to universal rules, not specialized knowledge you only need occasionally.</p>
I also kept retyping the same prompts for repeated tasks, like full doc review checklists. The same instructions, over and over, because I hadn’t saved them anywhere.</p>
These were minor annoyances, not blockers. But when I started documenting Claude Code’s features, I realized there were specific tools that fixed these friction points.</p>

What else Claude Code can do</h2> These are the tools you can reach for beyond prompting and CLAUDE.md</code>, and the kinds of technical writing tasks each one is best suited for. You don’t need to adopt all of these at once. Start with CLAUDE.md</code>, then add the others only when you feel friction.Slash commands: One prompt, saved forever</h3> Claude Code ships with a lot of built-in slash commands</a> worth exploring, like /init</code> to analyze your codebase and set up your configuration or /rename</code> to clean up session names. Beyond the built-ins, any prompt you type more than twice can be a custom command. They’re Markdown files you store in .claude/commands/</code> and trigger within a Claude session. Here are two light examples you might add to a technical writing toolkit: /review</code> (.claude/commands/review.md</code>), a full doc review checklist: Review this document for: 1. Style guide compliance (check CLAUDE.md) 2. Technical accuracy (do the code samples work?) 3. Broken links and references 4. Missing context (would a new user understand this?) 5. Structural issues (headers, flow, completeness) Categorize issues as: blocking, should fix, nice to have. </code></pre> /simplify</code> (.claude/commands/simplify.md</code>), for when you’ve written something too dense: Simplify this content for a developer who's new to the product. - Remove jargon or define it on first use - Shorten sentences - Cut anything not essential to the core concept - Add an example if it would help </code></pre> You can run /review</code> or /simplify</code> and Claude will run that prompt on the content you’ve been working on. (Full slash commands documentation</a>) Skills: Context that loads only when you need it</h3> I was using CLAUDE.md</code> for information that wasn’t applicable to every conversation. CLAUDE.md</code> eats into your token budget from the start of every conversation, so it’s not efficient to include information Claude may or may not need. Skills solve this. Skills give Claude specialized knowledge for specific types of work, like how to structure API docs versus tutorials or how to test code samples for different SDKs. They’re directories with instructions, templates, and reference files. At startup, Claude only loads skill names and descriptions. The full SKILL.md</code> file only loads when Claude decides from the task to use a skill. An API docs writing skill might look like this: .claude/skills/api-docs/ ├── SKILL.md # Instructions for API doc structure ├── template.md # Standard template with all sections └── examples/ ├── good-example.md # "Here's what we're going for" └── common-mistakes.md # "Avoid these patterns" </code></pre> The SKILL.md</code> file for this skill: --- name: documenting-apis description: Provides structure and examples for writing API reference documentation. Use when creating or editing API documentation. --- # API Documentation ## Structure Every API doc needs: Overview, Authentication, Endpoints, Error codes, Examples ## Voice More formal than tutorials. Assume reader knows HTTP basics. ## Template See [template.md](template.md) for the standard structure. Don't skip sections. ## Examples See [examples/good-example.md](examples/good-example.md) for reference patterns. </code></pre> The supporting files (templates, examples) only load when the task actually needs them. Claude discovers them through links in your SKILL.md</code>, so you can keep detailed reference material available without it eating up context until it’s relevant. Now your CLAUDE.md</code> can stay lean with only universal guidelines, while specialized context loads automatically when you need it. (Full skills documentation</a>) A note on Skills vs MCP: I’ve heard people say that Skills make MCP servers less useful, but that’s not the case. They serve different purposes. MCP is the connection to your data (Slack, GitHub, Google Drive). Skills are the instructions for how to use that data. For example: MCP server connects Claude to your GitHub issues</li> Skill says “When reviewing docs, check issues labeled ‘documentation’ for common user confusion”</li> </ul> You can have MCP without skills (you just prompt every time), but skills paired with MCP are way more powerful. Skills turn “I have access to this data” into “here’s when and how to use it.” (MCP documentation</a>) Subagents: Parallel work without losing focus</h3> Subagents are separate Claude sessions that run in the background with their own context. The main agent passes specific context to them, and they report back when done. You can spawn them ad hoc for one-off tasks. But you can also save subagent configurations as Markdown files with YAML frontmatter in .claude/agents/</code>. Once configured, they become reusable tools you invoke by name. I frequently use saved subagents for end-of-doc review: tech review, UX review, style review. They run in isolated context, which means I can kick off multiple reviews in parallel without competing for the main conversation’s token budget. Here’s an example of a subagent that checks for technical accuracy. Tip: you can use the /agents</code> slash command to help build these instead of doing it manually. Technical accuracy reviewer (.claude/agents/tech-reviewer.md</code>): --- name: tech-reviewer description: Verifies documentation matches actual code behavior and internal sources of truth. Use when reviewing documentation for technical accuracy or when code has changed. tools: [Read, Grep, Glob, Bash] --- You are a technical accuracy reviewer for documentation. When invoked: 1. Read the documentation being reviewed 2. Find the corresponding source code in src/ 3. Check internal docs in /docs/internal/ for canonical behavior 4. Verify every claim in the doc matches actual implementation For each section, report: - ✓ Accurate (code matches doc) - ⚠ Needs attention (minor discrepancy) - ✗ Incorrect (documentation contradicts code) Include specific line numbers and suggest corrections. </code></pre> When you’re ready for a review, kick this off by asking Claude to run it. You can keep working in the main thread as this and other subagents run in parallel (or you can take a coffee break), and when the subagent is done, you get a focused report back. (Full subagents documentation</a>) Subagents vs. skills vs. slash commands</h4> You might notice some overlap between these tools. Could the /review</code> slash command also be a subagent? Or a skill? Yes. In some cases, you might even want a slash command and a subagent that perform similar tasks. The difference is not what they do, but how they handle context and who triggers them. Tool</th> Context/isolation</th> Usually triggered by</th> When to use it</th></tr></thead> Slash command (/review</code>)</td> Full main conversation context</td> You (explicit invocation: /review</code>)</td> When you want Claude to consider the entire discussion and current project state.</td></tr> Subagent (review-agent</code>)</td> Isolated context (only what you/agent pass to it)</td> You (tell Claude to run it)</td> When you want to run tasks in parallel or keep your main conversation focused.</td></tr> Skill (review-skill</code>)</td> Loads specialized instructions on demand</td> Claude (infers from task context)</td> When you need structure, templates, or reference material without keeping it in context all the time.</td></tr> </tbody></table> Note: The triggering part can be more nuanced in practice. Claude can invoke custom slash commands, and you can explicitly ask Claude to use a specific skill. The table above covers the most common usage. You can use more than one of these for the same kind of task. A slash command is useful for quick, context-aware checks. A subagent is better for parallel or background work. A skill is the right choice when the work needs deeper, reusable guidance without bloating your main context. Hooks: Automation that runs at specific events</h3> Hooks are scripts that run automatically at specific Claude Code lifecycle events. Before a tool runs, after Claude sends a message, when a session stops. You set them up once and then forget about them. A useful one for technical writing: run your linter before any edit saves. { "hooks": { "PreToolUse": [{ "matcher": "Edit|Write", "command": "vale --config=.vale.ini $FILE" }] } } </code></pre> Now whenever Claude edits your work, linting runs automatically. You catch issues immediately instead of in code review. Hooks make checks deterministic. No relying on Claude to remember, no hoping it picks up the pattern. It just happens every time. Another useful one: a sound notification when Claude finishes a message. I tend to context switch while waiting for responses, and this tells me when to come back without constantly checking. { "hooks": { "Stop": [{ "matcher": "assistant", "command": "afplay /System/Library/Sounds/Glass.aiff" }] } } </code></pre> You can hook into a lot of events: session start, before context compaction, after Claude sends a message, on session stop. Anywhere you’d want to inject a check or action, there’s probably a hook point for it. (Full hooks documentation</a>) Plugins: Discover and share workflows</h3> Plugins</a> work in two directions: you can benefit from what others have already built, and you can share your own workflows with others. Using plugins others have built: To browse and install prebuilt plugins, run /plugin</code>. The default marketplace from Anthropic is automatically included, but you can add other community marketplaces</a> if you want to explore more. One plugin I use all the time is commit-commands</code> from the Anthropic marketplace. It auto-drafts commit messages based on my changes. Way more useful than “update skills docs” but also not something I want to spend brainpower on. Sharing your own: Once you’ve built up a collection of slash commands, skills, subagents, and hooks, you can package them as a plugin and share it via marketplaces</a>. Instead of every person figuring out the same workflows from scratch, you can hand them a plugin with everything already configured. A quick note on sharing: if you store your commands, skills, and agents in your project’s .claude/</code> folder and commit them to git, your team automatically gets them when they clone the repo. That works great for project-specific setups without needing a full plugin. (Full plugins documentation</a>) How these pieces fit together</h2> Here’s how I’m starting to think about it: Tool</th> What it’s for</th> When to use it</th> Example</th></tr></thead> CLAUDE.md</code></td> Universal context</td> Guidelines that apply to every task</td> Voice, formatting rules</td></tr> Slash commands</td> Repeatable prompts</td> Any prompt you use more than twice</td> /review</code>, /simplify</code></td></tr> Skills</td> Specialized instructions</td> How to approach specific doc types</td> API docs skill, tutorial skill</td></tr> Subagents</td> Parallel work</td> Reviews that can run in the background</td> Tech reviewer, code tester</td></tr> Hooks</td> Automatic guardrails</td> Checks you always want to run</td> Linter, sound notification</td></tr> </tbody></table> The goal isn’t to use all of these. It’s to know they exist so you can reach for them when they’d help. Where to start</h2> If you’re not using Claude Code yet: Install it by following the quickstart guide</a>.</li> Navigate to a docs repo and open Claude Code: cd your-docs && claude</code>.</li> Start chatting.</li> </ol> If you’re already using it but mainly just prompting: Run /init</code>: analyzes your codebase and sets up CLAUDE.md</code>, slash commands, and subagents based on what would be useful for your project.</li> Add universal guidelines to CLAUDE.md</code>: voice, formatting rules, repo structure.</li> Create one slash command for your most common task (maybe /review</code>).</li> Spawn a subagent for parallel reviews. When you repeat the same instructions, save it as a configuration.</li> </ol> You don’t need to set up everything at once. Add tools when you notice friction. What’s stuck with me most is how each tool solves a different problem around context, isolation, and automation. CLAUDE.md</code> gives you persistent context. Slash commands give you repeatable prompts. Skills give you on-demand specialization. Subagents give you parallelism. Hooks give you guarantees. I’m still learning which ones fit which workflows. But having them available means I can shape how I work instead of working around the limitations of a single interface. Code execution with MCP: separating context from computation Unknown — Fri, 14 Nov 2025 00:00:00 +0000 TL;DR: Traditional tool calling eats away at an agent’s context window. All the tool definitions and every intermediate result pass through it. Code execution gives agents a “scratch pad” workspace to process data outside the context window. Combined with MCP’s tool abstractions, this can significantly reduce token usage for data-heavy tasks. Check out the demo repository</a> for two examples you can run. I thought I understood MCP. It took me a while</a>, but I recognized that we need MCP as a standardized layer to best interact with external services. MCP gives LLMs structured access to APIs, providing a consistent interface instead of varying parameter names and request/response formats across different services. Then I read Anthropic’s post about code execution with MCP</a> and Cloudflare’s piece on Code Mode</a>, which talk about having LLMs write code to call MCP servers, and I got confused again. Why are we having LLMs write code around MCP servers when we wrote MCP servers so that… LLMs didn’t have to write code to interact with APIs? The piece that finally clicked: a key benefit of code execution is that it happens in an execution environment, not in the agent’s context window. MCP is still the standardization layer; generating Python wrappers for MCP tools makes them available as importable functions in the execution environment. Then, if the LLM has access to a bash tool that executes commands, it can write and execute entire scripts in its execution environment and only send the final answer back to context. As an analogy: traditional tool calling requires you to show every single calculation on your final exam paper: all the messy scratch work, the raw data, the transformed versions, crossed-out mistakes. You run out of space fast, and the grader (the LLM) has to wade through all that noise to find the final answer. MCP + code execution is like giving you scratch paper to process data and perform operations outside the context window. Then you just record the final result on your exam paper. Context window vs. execution environment</h2> The context window is the LLM’s working memory. It’s expensive (costs tokens), limited in size, and includes: System prompt </li> Conversation history </li> Tool descriptions </li> Tool results </li> The response being generated </li> </ul> The execution environment is where code actually runs. In development, it might be your laptop. In production, it’s typically a sandboxed Docker container or VM with limited filesystem and network access. It’s relatively cheap and can handle large data processing. Modern agent frameworks like Claude Code</a> and Cloudflare Agents</a> provide code execution environments built-in, but you can also achieve the same pattern by giving any LLM a bash tool that executes commands in your execution environment. The problem: your context window is getting bloated</h2> Tool calling is great for letting LLMs use APIs and external tools, but once you have a production system with large datasets and hundreds of tools, it quickly runs into a major problem: all the tool definitions and every intermediate result from every tool call has to pass through your context window. If you want to fetch 300 records, transform the data, and filter it with tools, all of the following must pass through context: The tool definitions </li> The raw data from the records </li> The transformed data </li> The filtered data </li> </ul> Each step consumes more of your expensive, limited window until you hit the limit or face substantial cost increases. Your agent also has to sort through all the results and its working memory is crowded with information that isn’t actually relevant for its final response. The solution: code execution in an execution environment</h2> Code execution gives agents a “scratch pad” workspace to process data outside the context window. It allows the agent to write code that calls MCP tools, process results locally in the execution environment, and only record the final result in context. MCP still handles the “How do I call this API correctly?” problem by giving LLMs structured tools instead of making them write raw API calls that vary from API to API. Code execution solves the “How do I process all this data without destroying my context window?” problem by giving the agent a workspace to process data outside the context window. Beyond data processing, code execution unlocks capabilities that are challenging with traditional tool calling. Things like polling for results and waiting, retrying failed requests, maintaining state across multiple operations, saving intermediate results and returning back to them later, all become possible through code. Production safety: guardrails for code execution</h2> Giving an LLM the ability to execute code is powerful, but requires guardrails. Three key practices: 1. Sandboxed execution: Run code in isolated Docker containers or VMs, not your production environment. Limit filesystem and network access or use something like the Cloudflare Sandbox SDK</a> or Daytona</a> for runtime isolation. 2. Human approval for destructive actions: Require approval for anything that costs money, modifies data, or communicates externally. For example, read_channel_messages</code> might auto-approve, but post_message_to_all_hands</code> should require review. 3. Thoughtful tool design: Don’t expose raw endpoints. Build high-level tools like schedule_meeting</code> (checks availability + books room) instead of list_users</code> / list_events</code> / create_event</code>. Prefer search-focused results (search_contacts(”Jane”)</code>) over giant dumps (list_all_contacts()</code>). The Anthropic article on writing tools for agents</a> covers these design principles in depth. When to use each approach</h2> Not every task needs code execution. Traditional tool calling works well for certain scenarios: Traditional tool calling works when: Tool results are small (a few hundred tokens) </li> You need one or two tool calls total </li> No data processing required </li> Results go directly to the user </li> </ul> Code execution excels when: Processing large datasets </li> Making multiple related tool calls </li> Filtering or transforming results </li> Polling or waiting for completion </li> Maintaining state across operations </li> </ul> Limitations and trade-offs</h2> Code execution has some trade-offs (and at least to me, this is such a new concept that there are a lot of things I’m haven’t thought through yet): Model capability matters: Code execution requires models with strong coding abilities. Not all LLMs write clean, correct code consistently. If your model struggles with code generation, the benefits diminish quickly. Test your chosen model’s coding capabilities before committing to this pattern. Non-deterministic code: Traditional tool calling shows you which tools are called, with what arguments, and what they return. Code execution is opaque; the LLM generates code dynamically, and that code can be different each time for the same prompt. You don’t know what it will write until it writes it, and once it’s executing in the environment, you lose visibility into the step-by-step operations. Security surface area: Executing LLM-generated code introduces risks beyond traditional tool calling. Even with sandboxing, you need to worry about resource exhaustion, unintended file system access, infinite loops, and malicious prompt injections that generate harmful code. The guardrails mentioned earlier are necessary but not sufficient; you need ongoing monitoring and careful tool design. Complexity: Your system needs to manage an execution environment, handle code execution errors gracefully, and potentially deal with environment setup and dependencies. Traditional tool calling has a simpler operational model. Despite these trade-offs, for data-heavy tasks where token costs and context window limits are real constraints, code execution can provide significant practical benefits. The mental model shift</h2> Agents are more than LLMs with tools. They’re LLMs with tools and an execution environment. The context window is for reasoning. The execution environment is for working. When you structure agents this way, you get: Significant reduction in token usage (varies by task, but can be substantial for data-heavy operations) </li> Natural patterns for polling and async operations </li> Ability to process large datasets </li> Stateful computation without context bloat </li> More predictable results by executing deterministic code rather than tracking state in context </li> </ul> Code execution can improve reliability by reducing the need for the LLM to track state and aggregate data across multiple tool calls in context. By having the agent write deterministic scripts that run in the execution environment, you get more predictable and complete results. Note: The effectiveness of this approach depends significantly on the model you use. Models with strong coding capabilities will write better, more efficient code. The token savings also vary based on your specific use case—tasks involving large datasets or many sequential operations see the most benefit. MCP provides the standardized tools. Code execution provides the workspace to use them efficiently. Together, they enable agents that are both capable and cost-effective. Try it yourself</h2> I created a demo repository</a> with two examples you can run and play around with: Traditional tool calling: Shows how intermediate data fills the context window </li> Code execution: Shows how processing happens in the execution environment </li> </ol> Both examples include token usage metrics so you can compare the approaches. Note on token variability: Token usage varies between runs, even with the same task and model. In my testing with this demo, I’ve seen the following results: Traditional tool calling: Typically 50,000-73,000 tokens </li> Code execution: Typically 9,500-10,000 tokens </li> Token reduction: 80-87% depending on the run </li> </ul> To run the examples: `# Clone this repo git clone https://github.com/sarahcstringer/mcp-code-demo cd mcp-code-demo Install dependencies</h1> pip install -r requirements.txt Set up your Anthropic API key</h1> cp .env.example .env Edit .env and add your ANTHROPIC_API_KEY</h1> Run the traditional tool calling example</h1> python examples/traditional_tool_calling.py Generate MCP tool wrappers for the code execution example</h1> python generate_wrappers.py Run the code execution example</h1> python examples/code_execution.py`The examples use Anthropic’s Claude Haiku 4 model with the Anthropic SDK, but the patterns work with any model provider. Next steps</h2> Read the Anthropic post on code execution</a> </li> Learn best practices for writing tools for agents</a> </li> Check out Cloudflare’s Code Mode explanation</a> </li> Learn about MCP</a> </li> </ul> On agency, abstraction, and AI-assisted coding Unknown — Tue, 04 Nov 2025 00:00:00 +0000 I saw a screenshot of this email about AI-assisted coding and agency, and it sparked a lot of thoughts for me. The core argument was that AI-assisted coding trades long-term mastery for short-term outcomes, and that represents a loss of personal agency. Agency, in this framing, means having the courage to struggle through problems on your own, stretch beyond your current abilities, and build the kind of deep understanding that only comes from doing the hard work yourself. It’s a thoughtful argument, but I don’t experience it that way and wanted to share my perspective. The abstraction layer argument</h3> When I see arguments that using AI means you don’t actually know what you’re doing or haven’t built something yourself, I wonder how it maps to previous advances in tooling and technology. To me, AI coding tools feel like another abstraction layer. Just like I don’t need to understand exactly how the file system retrieves data from disk to write meaningful programs, I don’t need to hand-craft every line of code to build something real and useful. When higher-level languages like Python became popular, some low-level programmers dismissed them as “not real programming” because they abstracted away too much control. When React rose in front-end development, other developers complained that newcomers no longer understood the DOM or how browsers actually work, but those new developers were still building interesting, useful products. The waves of abstraction spark similar anxieties that hiding complexity will erode understanding and craftsmanship. Even if some people feel that way, many of us recognize that higher-level languages didn’t eliminate the need for problem-solving or understanding. Instead, they shifted where we spend our cognitive energy and allowed us to do more by building on top of the existing technology. I understand that calling AI “an abstraction layer” might be a bit of a stretch. But I still think of it similarly. With AI assistance, I’m learning to orchestrate multiple tools, debug unexpected behavior, and think through architecture in ways I couldn’t before. I’m learning about different frameworks I hadn’t previously explored. The thinking hasn’t disappeared, as far as I’m concerned; it’s just operating at a different level. And, I’m actually able to ship the things that used to live only in my head. Where agency actually lives</h3> One of my friends won’t use AI to draft Slack messages because he wants his words to genuinely come from him. It’s important to him that his reasoning process be fully his own. I get that, but also… not. When I use AI to help articulate an idea, the thinking is still mine. The struggle to figure out what I’m trying to say, the iteration on tone and clarity, the decision about what to include or cut still feels like me. Beyond that, I’m actually thinking more, not less. Ideas that would have stayed trapped in my head, half-formed and unexplored, now make it out into the world. And once they’re out, I keep iterating on them. I keep expanding, refining, and connecting them to other ideas, and connecting with more people with diverging thoughts. AI is removing the friction that used to stop me from creating things in the first place. That sense of magic David Beazley describes, of “commanding complexified sand into performing stunts,” hasn’t diminished. If anything, it’s intensified. Different tools for different people</h3> I don’t expect to convince anyone who experiences AI assistance as a loss of agency. Different people relate to their craft differently, and that’s completely legitimate. One thing does bother me about this broader narrative around ownership and mastery, though. I don’t think this is what the author intended, but I dislike the framing that tells people they don’t actually own what they feel they’ve built. When someone uses AI tools to create something they’re genuinely proud of—something they designed, debugged, and shipped—telling them “that doesn’t count” or “you didn’t really build that” feels unnecessarily dismissive. It’s a kind of gatekeeping that decides in advance what “real” mastery looks like and who gets to claim ownership over their work. I want folks to know that using AI tools doesn’t mean I’m not thinking, not learning, not creating, or not growing. It means I’ve found a set of tools that let me finally build things I’ve been dreaming about for years, and the learning curve is in figuring out how to work with these tools effectively, which is its own kind of mastery. Technology has always been about layering abstractions. Some people prefer working closer to the metal, and others embrace new layers that expand on that. Both approaches have value. Both require agency, courage, and continuous learning. I don’t feel that mastery disappears when tools evolve, it just evolves with them. What’s your experience? Do you find AI tools expand or constrain your sense of agency as a developer? Testing docs IA with AI agents Unknown — Mon, 20 Oct 2025 00:00:00 +0000 A few days ago, I posted about using the Claude Agent SDK to play the Wikipedia game</a> as a way to think about docs navigation using an LLM. This weekend I worked on an agent that would navigate my actual docs site with real user questions, and immediately started getting value out of it. Link to the GitHub repository if you want to try it out.</a> Here’s what I built, what it found, and why I’m excited about it. The unexpected gift of AI assistants</h2> Before we had an AI Assistant on our docs site, understanding what people needed was mostly inference. I looked at website analytics that said little about the user experience, combed through support tickets to see what hurt enough to escalate, and crossed my fingers that users would submit feedback if they felt strongly enough about the docs. Now users just tell us in their own words what they’re trying to do. Each conversation is a little case file: the goal, the vocabulary they use, what they already know or don’t, where they get stuck, and what they assume exists. A request for teams building AI docs assistants</h3> If you’re implementing an AI assistant for a docs site, please include conversation IDs, user or session IDs, and timestamps in your exports. Most logs are still unthreaded Q&A pairs. Even minimal session data would help see things like “they asked about authentication three times before contacting support,” or “people who ask about X usually return asking about Y.” The raw questions are already useful; basic session observability would be transformative. The setup</h2> I exported the AI Assistant logs, then wrote a script (persona_extraction.py</code>) that asks Claude to analyze the questions, cluster them into realistic personas, and preserve the exact phrasing people used. You could also feed the same script any source of real user input: a CSV of support tickets, survey responses, feedback forms, or search queries. The point is to start from real language, not imagined tasks. A second script (docs_navigation_test.py</code>) does the navigating. Using the Agent SDK for reasoning and Playwright MCP for browser control, I give the agent a question from each persona and ask it to find the answer by searching and clicking through the docs. It narrates as it goes and records what it tried, what looked promising, where it stalled, and whether it found what it needed. I tailored the prompt to have the agent mimic an impatient developer: favor the first few search results, skim, give up after a handful of misses, and show frustration when paths don’t work. Constraining behavior</h3> Because I’m testing navigation, not general problem-solving, I restricted what the agent can do. The SDK exposes a lot of tools, but for this experiment I only allowed Playwright’s browser actions: navigate, read, click, type, manage tabs. If it tries anything else, the hook blocks the action and explains why. You can fence it tighter if you want: limit it to your domain, disable tab switching, or log blocked attempts. The point is to model the user you care about. If the question is “can they find this without leaving the docs?”, enforce that. What the first few runs revealed</h2> When the IA is solid, the traces look like the paths you hope users take. Faced with “How do I access a file uploaded from a file input block in state?”, the agent searched “file input state,” landed on Add files to Writer Cloud, picked up how state variables bind, pivoted to Agent state, and had a complete answer in under two minutes. When there’s a hole, it shows immediately. The question “is there a multimodal model that can process video files?” sent the agent on a tour around unhelpful docs. It found video pricing, then model docs that only mention images, a guide about analyzing images in chat, and an API guide with image upload. We charge for video, but we don’t document how to use it. Sometimes the docs hint at an answer without saying what you actually need. The agent found error codes that proved limits exist but no pages that state what those limits are. Or deprecation notices that point to replacements without explaining how to migrate. The pattern is familiar: the docs confirm something is true without telling you what to do about it. And sometimes the gap is an audience mismatch. A sales rep landed on the developer portal asking about “highest-value propositions.” The agent searched, came up empty, and concluded, correctly, that it was the wrong site. That’s not a docs flaw so much as a routing issue, but it’s still a signal: people in the wrong role are ending up here. You can ignore it, redirect them, or track the frequency and decide later. A bug and a very frustrated agent</h3> While testing the agent, I wrote a bug in a loop that kept forcing the agent to continue after it had decided to stop. After ten failed attempts it snapped: “Look, I’m role-playing an impatient user who’s been searching for 10+ steps already. A real user would have QUIT by now. I’ve already given my answer multiple times.” And the loop continued… “I’m DONE. An impatient user would have rage-quit ten steps ago.” When nudged again: “STOP. I’m role-playing an impatient user…” Somewhere in the theatrics, it found something true: we don’t document file size limits. Ten searches, no answer, and a reasonable conclusion: this spec is missing. I’d contact support or give up. Why this worked better than I expected</h2> The realism helps. The agent doesn’t grep for strings; it searches, clicks, follows promising links, gives up on dead ends, and explains why. It also doesn’t always take the same path. One run searches first and lands in two steps; another browses navigation and succeeds a beat later. Good IA should support both. It surfaces confusion as well as absence. Answers that exist but are buried too deep, split across pages, or written in jargon that users don’t know show up as misses. An unexpected bonus: functional issues</h3> Because the agent actually drives the UI, it also stumbles over functional bugs. When I simultaneously ran two sessions that fought over browser resources, one test couldn’t open the search dialog at all. The agent’s summary was what you’d expect from a blocked user: I can’t start because search won’t open. It’s not a replacement for proper end-to-end tests, but it’s a surprisingly handy side effect: while testing findability, you also discover broken interactions. What it doesn’t replace</h2> This isn’t user research. The traces won’t tell you why someone is confused, what else they tried, or if the docs actually helped them implement a solution. It maps friction, not emotion. Also, the default agent is too patient and lacks the reader’s motivation. Real people skim, guess, and bail quickly. The agent politely keeps going. What I wasn’t testing, and what I learned anyway</h2> I wasn’t trying to test whether our docs were “agent-friendly.” That’s a different problem: how content gets chunked, embedded, or retrieved through RAG pipelines or search indexes. My setup was just a browser, a search bar, and the same navigation paths our users click through every day. But watching the agent struggle or succeed still made something clear. The pages it found fastest weren’t “optimized for AI”; they were just well-written. Headings with context (“Configure your agent to use environment variables”) worked better than vague ones (“Configure”). Links that explained their purpose guided it cleanly (“See OAuth authentication to connect your app”), and plain language (“video”) outperformed our internal jargon (“vision”). So this wasn’t a test of agent-readability; it was a test of how well a machine could survive the same IA humans do. The bottom line</h2> AI assistants are changing how people interact with documentation. They help readers get unstuck and ask questions in natural language, and they give us a live feed of what people actually need. That visibility makes it possible to test documentation in ways we couldn’t before: real scenarios, real language, real task completion. Good IA still matters, maybe more than ever. Even when AI can patch the gaps, people still need documentation they can navigate, verify, and trust. Can AI help test IA? A Wikipedia game experiment Unknown — Fri, 17 Oct 2025 00:00:00 +0000 I’ve been wondering for a while how to test or verify information architecture when I can’t actually sit down with users and watch them navigate the docs. User research is expensive, time-consuming, and hard to do at scale. Traditional metrics don’t feel like they capture the right things either. I started wondering: could I build an agent that behaves like a user navigating a site? Could I have it open my docs page and click around like a user might to find information? Before trying it on my own docs, I wanted to see a sample of an AI navigating from one place to another with a goal in mind. So, I had it play the Wikipedia game</a>. It was interesting to see the paths the agent took, but watching the agent play and then playing the game myself ended up reminding me of a bunch of things about how humans move through information in ways agents wouldn’t. The Wikipedia game</h2> There are a few variations of the game, but the idea is: start on one page and try to reach another by clicking only internal links and not using the back button. For example, go from Dog</a> to Mathematics</a> using only links between pages. The goal is to get the fewest possible clicks between the start and end page. That feels somewhat similar to what users do when navigating docs (at least when they’re not using Google or AI as intermediaries): getting from “I have this problem” to “I found the answer.” Building the agent</h2> The setup was pretty quick. I used the Claude Agent SDK</a> and, because Wikipedia is all static HTML, all I needed to do was let the agent run curl</code> commands to fetch pages. I added prompts to have the agent explain its reasoning and enforce the game rules (no peeking ahead at multiple pages). Here’s the full code if you want to try it out.</a> Watching the agent navigate</h2> I tried a few sample runs. I also cross-referenced Six Degrees of Wikipedia</a>, which calculates all the shortest possible paths between pages, to see how close the agent was to optimal. Sometimes the paths the agent took were the shortest possible paths, sometimes not, but they were always logical. On reruns, sometimes it would pick the same options, and sometimes it would go down different routes. Here are a few example runs: Yo-yo → Keynesian economics (shortest: 3 clicks, agent: 4 clicks) Yo-yo → World War II → Franklin D. Roosevelt → New Deal → Keynesian economics </li> Reasoning steps: “Keynesian economics became highly influential during and after WWII 2” </li> “FDR implemented the New Deal, which was heavily influenced by Keynesian economic principles” </li> “The New Deal represents one of the most significant applications of Keynesian economic principles in practice” </li> </ul> </li> </ul> Cactus → Algorithm (shortest: 2 clicks, agent: 3 clicks) Cactus → Fibonacci numbers → Euclidean algorithm → Algorithm</li> </ul> Skateboarding → Beowulf (shortest: 2 clicks, agent: 3 clicks) Skateboarding → Culture of the United States → American literature → British literature → Beowulf</li> </ul> The agent was methodical and patient. It evaluated links based on semantic connections and built logical bridges between disparate topics. Then I played the game myself</h2> I hadn’t actually played yet. So I tried Tea</a>** → **Norse mythology</a>. And immediately realized: I had no patience for this. The Tea page, like many Wikipedia pages, is huge, with hundreds of links. I skimmed the first paragraph, got overwhelmed, and hit Ctrl-F for “myth.” I found: “Chinese legends attribute the invention of tea to the mythical Shennong</a>.” I’d never actually heard of Shennong before, but the context around the link guided me to the right place. My path: Tea → Shennong → Mythology → Norse mythology When I had the agent play, its path was: Tea → Germany → Germanic peoples → Norse mythology. The gap between us</h2> When I compared my experience to the agent’s, I noticed a few key differences. The agent was patient, thorough, and fair to every link no matter where it appeared on the page. It built careful semantic bridges. I was impatient and wanted to quit almost immediately. I skimmed, used shortcuts, and clicked based on hunches. I could try adding time limits or other constraints to the agent, but the initial contrast reminded me of something I sometimes forget when I’m deep in docs work: I’m not designing for someone who’s going to read every page top to bottom. I’m designing for me on a bad day with a production outage to fix. What the experiment reminded me about IA for humans</h2> 1. Readers skim and use context clues I didn’t read the whole Tea article. I didn’t carefully evaluate every link. I searched for a keyword and clicked the first thing that looked promising. The context around the link—“the mythical Shennong”—told me what I needed to know. I didn’t need to understand who Shennong was. The word “mythical” was the signal. In docs, we often write “See Authentication” with no context. No hint about what kind of authentication or why you’d click that link. Users have to click blind or skip it entirely. More information about why someone would want to go to a page, like “See OAuth authentication to connect your app to our API,” helps orient readers. 2. Position matters I looked at the first paragraph of the Tea page. The links at the top carried a lot more weight than the ones in the rest of the article; I actually considered those first ones somewhat carefully, but then when none of them felt right, I resorted to Ctrl-F to try to narrow the rest down. The agent read everything. Real users don’t. If the most important information isn’t in the first one or two paragraphs, you’ve already introduced friction. 3. Background knowledge creates shortcuts and gaps The agent has a huge background of knowledge that real users don’t. It can see a name like Carl Linnaeus</a> and know immediately who that is and how his page might relate to the overall topic. (I had no clue, but he was a mathematician and that’s what Claude used to get to the Mathematics page.) A real user with knowledge gaps would get stuck or gloss over the link, not knowing if it was relevant. Assuming all users know what “OAuth” or “webhook” or “idempotent” means without context is designing for experts, not for all the people who might need help. 4. Motivation changes everything The agent kept going no matter how hard the path. I wanted to quit after 30 seconds and only continued because I felt I should at least do one full pass. Users without motivation, or with competing priorities, won’t slog through complexity. They’ll bail and find another alternative, or just give up. (One positive thing here: AI tools are great at helping people push past these barriers that would normally stop them. Users can ask questions, get answers that speak to them, and keep moving forward even when the docs IA or content isn’t perfect.) 5. Different mental models, different paths When I played the game, I found one logical path. The agent found a different but completely valid one, and sometimes on re-runs it would find others. Real users would find dozens more. If your IA assumes everyone navigates the same way, like always task-based (“How do I…?”) or always concept-based (“What is…?”), you’re designing for one mental model and abandoning everyone else. What this means for docs</h2> As a next step, I want to point an agent to my docs and give it tasks that actual users face, like, “how do I get an API key,” or “I’m getting this error in production, how do I fix it?” Even though it won’t get at the actual human experience, it might be able to help me highlight paths that are missing or could be improved. But in the short term, the experiment reminded me of things I can look for right now, without running any agent: Do my links have context? Not just “See Configuration” but “See Configuration to set your API timeout values” </li> Are the most common paths visible immediately? Or do users have to scroll, search, or guess? </li> Am I designing for patient agents or impatient humans? Users will use Ctrl-F, skim, and quit fast. Design for that. </li> Am I assuming knowledge users don’t have? Every unexplained term is friction. </li> Does my IA support different mental models? Task-based, concept-based, troubleshooting-based; different users navigate differently. </li> </ul> The way users navigate docs is rapidly changing. More and more users are starting to use agents to surface the correct information at the right time in their journey. But solid IA within docs is still critical. Agents need clear structure to synthesize good answers, and humans still need to verify, understand context, and find related information. Watching an agent navigate patiently and methodically, and then experiencing my own impatience and shortcuts, brought back these important concepts. The Exploratorium boulder: why enthusiasm isn't always contagious Unknown — Fri, 10 Oct 2025 00:00:00 +0000 Seven months ago, I was skeptical of AI. I had a colleague who was wild about it, constantly building AI tools that no one else on the team used. His enthusiasm actually made me less inclined to try it. I assumed I already knew what he was doing with AI without ever trying it myself. Fast forward to the present day: I’m the enthusiastic one now. I’m all-in on Claude, Cursor, AI-assisted docs workflows, the whole thing. A few weeks ago, I sat down with another former colleague to show her what I’ve been using. We both worked with the same AI enthusiast colleague, and since leaving the team, I’d become super “AI pilled” while she hadn’t done much to adopt it. So, I thought I’d show her what was actually working for me and how things had changed. I asked if she was working in docs-as-code. She wasn’t. “Ah, okay,” I said. “So none of these things will really work for you.” And then I proceeded to show her all of them anyway. There were a few things she could use, like Claude for drafting or asking questions. But while I was talking, I kept feeling like I wasn’t making any headway. She was polite but not convinced. And then I realized that I was doing exactly what my old colleague had done to me. I was showing her the tools that worked for me, in my workflow, without understanding where AI would actually make sense for her. I never asked what her pain points were. I never asked what she loved or hated about her current process. I just demoed a bunch of stuff that didn’t fit her world. For example: I love using AI to generate a messy first draft that I can shape and refine. But maybe that’s her favorite part of writing. Maybe what annoys her is something totally different, like formatting, research, or keeping track of updates across multiple docs. I didn’t ask. I just showed off. The Exploratorium boulderMy favorite exhibit at the Exploratorium</a> when I was a kid was a massive cement cylinder hanging from the ceiling, surrounded by a circle of tiny magnets on strings. The cylinder had a strip of metal around it, and the goal was to throw your magnet onto the metal, connect it, and then pull the boulder in your direction. If you pulled hard, the magnet immediately detached. You got nowhere. The key was to connect the magnet and then do the tiniest possible pull. Not even a pull; more like a whisper of movement. Slowly, gently, your pulls can get stronger and the whole boulder starts swinging your way. I loved watching kids yank the magnets over and over and then give up. Then I’d sneak in, knowing the secret. Throw the magnet, barely touch the string, and do the gentlest tug. It felt like I could speak its language, like I was working with the cylinder instead of fighting it. (As an adult I realize there was probably a poster right next to the exhibit explaining exactly how it worked and what to do, but still, it felt like a secret.) Over the years, I found myself remembering this experiment in other contexts, especially psychology and human interactions. You can’t yank someone to your perspective immediately. You have to meet them where they are, truly understand where they’re coming from, and move from there, together. Learning AI and the 10 hour rule The same feels true for AI adoption: you have to try it yourself to find what works for your flow. Ethan Mollick suggests that you need to spend about 10 hours just using AI</a> for work or fun to really understand it. For about 10 hours, just do stuff with AI that you do for work or fun. Poke it, prod it, ask it weird questions. See where it shines and where it stumbles. Your hands-on experience will teach you more than any article ever could (even this long one). You’ll figure out a remarkable amount about how to use AI effectively, and you might even surprise yourself with what you discover. Somewhere in those 10 hours, most people hit a moment where something just clicks. But even tech writers on the same team aren’t going to have the same aha moments. My first win with AI was realizing Claude could generate a diagram for me in 30 seconds that would’ve taken me 20 minutes in Lucidchart. That tiny win opened the door for everything else. But someone else might not care about diagrams at all. Maybe their moment is rewriting a dense paragraph, generating test data, or summarizing a long Slack thread. You can’t shortcut someone else’s discovery process. You can’t demo them into belief. What I should have doneInstead of showing my workflow, I should have suggested we sit down and work together. I should have asked: What are you working on right now? </li> What’s annoying you about it? </li> Want to try throwing this problem at Claude and see what happens? </li> </ul> And then we could have experimented together. Maybe it would’ve worked, maybe it wouldn’t. But at least we’d be starting from her reality, not mine. Late in our conversation, she mentioned she was trying to build an agent and struggling to craft the right prompt for it. I said, “Oh, I’d just ask Claude to help me write the prompt for the agent.” That seemed to land. Not because it was the coolest AI trick I showed her, but because it directly addressed a problem she was facing right then. That was the magnet connection. Not “here’s all the ways I use AI,” but “here’s one small thing that solves your actual problem today.” I think this approach is especially important for skeptical folks. Once you’re already onboard with AI, sure, demos are great. But if you’re still unsure? A demo just feels like someone showing off. A working session feels like collaboration. The irony is that I needed someone to meet me where I was seven months ago. I wasn’t ready for “here’s my whole AI-powered setup.” I needed someone to say, “What’s annoying you right now? Let’s try one small thing together and see if it helps.” And really, I just needed space and tooling to poke and play. Meeting people in their worldWhen we throw out our most brilliant, complex use cases through demos, that’s not where the magic happens for most people. The transformative moments aren’t in the flashy stuff. Instead, they’re in the mundane examples that meet people in their real struggles. The meeting notes summarizer that means you’re not missing anything important. The health transcript note taker that saves nurses from burnout. The prompt helper that unsticks you when you’re building something. Ethan Mollick has also made this point often: focusing only on “transformational” use cases obscures the fact that small, meaningful ones are already changing work in quiet ways. He jokes</a> that we’ll know the big AI labs finally get it when they stop having “Dev Day” and start having “Non-Technical Manager Day.” The whisper, not the yankWhen I’m excited, I lead with my favorite, most impressive workflows. But those aren’t anyone else’s entry points. They’re the things I love because I’ve already done my 10 hours and discovered what fits my workflow. If you’re excited about AI (or any tool, really), it’s tempting to show people everything you’ve figured out. But excitement isn’t always contagious. Sometimes it’s overwhelming. So here’s what I’m trying to remember: ask first. Listen. Find the tiny magnet connection point. Then whisper. The boulder will start moving with you, together. Docs metrics and the stories we tell ourselves Unknown — Tue, 23 Sep 2025 00:00:00 +0000 My roles prior to developer education were in site reliability and infrastructure. When I joined my first docs team, I immediately asked where our dashboards lived. The answer: “There’s an old Looker Studio dashboard pulling from Excel sheets that’s been broken for a few months.” In SRE, observability was everything. I could see at any given time how the system was performing, and when I fixed something, I could immediately see it. I could dive into a flame graph or a trace, identify a bottleneck, apply a fix, and see the metrics move. Instant, concrete feedback. Docs metrics are not like that. The metrics gap</h2> I fixed the broken docs dashboard, excited for numbers to roll in and describe our documentation system. I started seeing page views, user comments, geographic information from users, time on page. For a while, I thought we were in business. But then: nothing. No one made decisions from the data. No one even looked at the charts now that they were fixed. And slowly I realized that the numbers I was tracking didn’t actually answer the questions I had. Leadership leaned on traditional top-of-funnel metrics like pageviews and traffic sources to quantify docs’ contribution to pipeline. Meanwhile, ICs wanted to know: Are the docs actually helping? Are users able to find what they need? Where are people getting lost or dropping off? Those are the answers that would let us improve the docs, but they’re harder to measure. I do understand the value in some of those traditional metrics. They can help justify new headcount, guide prioritization, or demonstrate business impact when you need to make a case for resources. They have their place. But they don’t answer the questions that keep driving me. I started looking for new metrics that would give me a better understanding of our docs system. I read books, listened to podcasts, watched conference talks. And I always felt dissatisfied with the answers I received. Really, what I want most to know is: Is this helping anyone? </li> How could it be better? </li> Am I any good at this? </li> </ul> Eventually, I stopped looking for answers. According to the 2025 State of Docs report</a>, 39% of docs teams don’t measure anything at all. For a while, I’ve been one of them. But that doesn’t feel right either. In chasing the answers I wanted as an IC, I stopped measuring altogether, and in doing so, I dropped signals that could still be useful for the team and for leadership. Numbers and the stories we tell ourselves</h2> A friend in Business Intelligence once told me his role was to be “less and less wrong each day.” That line stuck with me. In messy human systems, metrics aren’t truth. Take exit pages or bounce rate: in marketing, high exit and bounce is always bad. In docs, it could mean a user gave up in frustration, or that they found their answer and left satisfied. Same number, opposite stories. The data we do have is incomplete: scattered across tools, missing offline use, blocked by cookies, and now distorted by AI as more people pull answers from assistants instead of visiting the docs. The numbers don’t decide for us. They only frame possibilities. It’s up to us to interpret, connect, and tell the story. Who knows if that story is the right one? We’re just trying to inch towards some truth. I once heard a talk from someone who built dashboards for pro League of Legends players, and he said one of his biggest takeaways was that the teams were much less data driven than they thought. They used intuition first, then used data to back up their story. I think that’s often how docs metrics end up working too. What I’m actually going to do</h2> For a while, I let the lack of perfect measures paralyze me. I wanted the clean dashboard, the SRE-style flame graph of docs. But chasing perfect meant I wasn’t measuring anything at all. My manager had little way to demonstrate my work beyond anecdotes, and I had no way to point to trends, prioritize projects, or make the case for what needed attention. Then I saw this image and it really hit me: the crooked ladder still gets you where you need to go. So I decided to build my own imperfect ladder using tools I’d overlooked or dismissed before. Sarah Moir’s</a> work on docs analytics has been grounding here. Her conclusion in her post “Where to start with analytics for documentation</a>”</a> really resonates: “Analytics are intimidating, but remember, they’re fuzzy too. Fuzzy and imperfect proxies for what we really want to know: Is our documentation helping?” Instead of chasing one comprehensive solution, I’m taking an incremental approach: starting with basics I’d written off as insufficient, then building up as I learn what actually matters. Step 1: Track what you can</h3> The latest State of Docs report</a> shows that the most commonly tracked metrics are page views, user feedback widgets, internal search terms, and time on page. So here’s what I’m actually going to start with as my baseline: Page views: just to see what content gets the most traffic. </li> Helpful / not helpful votes: at least a signal when something isn’t working for someone. </li> Internal search terms: especially the ones that return no results, since that shows where the gaps are. </li> Time on page: still ambiguous, but maybe useful if I can connect it to changes I make. </li> </ul> There are other numbers I’d like to chase down too, specifically time from signup to first API call, or to a certain level of spend. Those can be powerful indicators of activation and product adoption. But they only make sense in context: if a lot of onboarding is handled by Solutions Engineers or CSMs, the docs might play a smaller role. I can probably filter for self-service accounts to get a cleaner signal of how documentation influences those timelines. Step 2: Control what you can measure</h3> While I can’t sit next to users and hear what they think about our docs, I can define what quality means to me and track against it. External metrics live with the user and are hard to measure directly. I can balance those with internal metrics where I create a checklist for quality I can control: Does the doc have troubleshooting steps and next steps? </li> Is the code complete and copy-paste-ready? </li> Is the language clear and consistent with our style guide? </li> Is it written at the reading level we expect for all our docs? </li> How many linting errors, warnings, and suggestions are there? </li> When was the doc created and last updated? </li> </ul> These are metrics we can actually measure within our docs set. It’s not the same as knowing if the doc truly helped someone, but it gives me a baseline of quality I can control, and can give some sense of comfort that we’re at least meeting the quality benchmarks we’ve defined. It can also give us more information about which docs need attention, especially when paired with other site analytics. Step 3: Watch for new opportunities</h3> I’m really intrigued by AI assistants. Han Wang from Mintlify says 40% of users on their docs sites ask the assistant something before clicking around</a>. Lots of people are now navigating through an interface where they tell you exactly where they’re starting and what they want to know. You can even watch how they continue after being presented with information. That feels like the closest thing to observability for docs I’ve seen. Context in the questions people ask, how they navigate, how we interpret the data itself. I’m hopeful about this direction, but I’m also braced for disappointment. I’ve chased a lot of metrics that promised clarity and delivered noise. Step 4: Keep looking, keep learning</h3> Finally, I’ll look at these metrics regularly. A few times a week, I’ll check the data, get a baseline, and start weaving a story from what’s there. The real commitment is to keep looking, comparing, and asking better questions. On a small team, I’ve never really needed to justify my time with metrics. But I can see the business value in them: they give me a point of view I can share with leadership, a way to frame priorities, and a starting place for conversations about where docs fit into bigger goals. Conclusion</h2> I may never get the beautiful, real-time, unified Grafana dashboard for my documentation that I crave. But I can still try to use what data we have to reduce uncertainty, guide decisions, create a shared vocabulary for talking about impact, and tell better stories about what’s working and what’s not. I’m trying to embrace that the real work isn’t finding the one metric that says it all; it’s staying curious, asking better questions, and building a system that gets a little less wrong each day. How onboarding a human made my AI smarter Unknown — Sun, 21 Sep 2025 00:00:00 +0000 The most recent team I joined had a set of docs but had never had a dedicated technical writer. It was a collection of information that had grown organically over time without style guides, linting, or any consistent voice. I had every intention of coming in and setting up the foundational guides and best practices in my first month. But then I got sucked into fighting fires and pushing out new docs. There was always something more urgent or rewarding than setting up those guardrails. Plus, I already knew the style I wanted and was the only one who had to conform to it. Six months later, a new person joined our team, and I realized I had nothing set up for her to onboard onto the docs. I became the bottleneck, dropping nitpicky PR comments like “actually, we use sentence-style capitalization…” instead of setting her up to succeed. I scrambled to codify our standards, and was surprised with the effect: getting those base guides in place didn’t just help my human teammate, but it elevated my AI tools, too. CodeRabbit</a> reviews became so on point and Cursor</a> generated drafts in our correct voice following the correct templates. I’d heard the term “context engineering,” but it wasn’t until I reflected on this process that it clicked: whether human or AI, teammates need context. The same onboarding materials that help humans understand how we do things here also give AI the information needed to perform at a higher level. What I did</h2> Here are the pieces I put in place that made a difference: Style, voice, and tone</h3> The biggest lift was that I finally wrote down all the style rules I’d been carrying in my head. I set Google’s</a> as the base style guide to follow, so I didn’t have to document every single piece </li> I wrote down any overriding preferences for capitalization, terminology, and phrasing with real before/after examples </li> I stored everything in a STYLE-GUIDE.md</code> for a human-readable format and in .cursor</code> project rules</a> so my AI tools could reference them directly </li> </ul> With this in place, Cursor was able to almost one-shot new doc generation and CodeRabbit, which also can access Cursor project rules, started giving truly helpful PR reviews that saved me from my own carelessness many, many times. Linting and automation</h3> I finally set up Vale</a> to catch any spelling or style errors before they reached production: I used AI to help me tune Vale’s rules to cut down false positives while catching real problems </li> I looked at established projects that use Vale like Grafana</a> to help figure out best practices </li> I configured Vale to flag the specific things that matter to our docs, like terminology mismatches, formatting quirks, tone shifts </li> I squashed all the errors in our docs so we had a baseline and then added Vale to our CI to catch any new errors </li> </ul> Now Cursor is able to see and fix Vale errors as it edits docs, and the human author gets notifications about errors as PR comments, so we don’t introduce new ones. Process documentation</h3> I wrote down not just what I prefer, but how I actually work. I documented my editorial workflow: step-by-step how I approach drafts and reviews, what I look for first </li> I captured context about our audience and goals that I’d never articulated before </li> I created templates that reflect how I actually structure different types of docs </li> I pointed Cursor to the source of truth repos so it could verify information from the source like I would </li> </ul> Now Cursor and CodeRabbit are able to look holistically at the docs I create and consider additional context the audience might need when approaching them. What I learned</h2> I didn’t expect AI to get better just because I onboarded a human teammate, but once I wrote down our rules and automated our guardrails, AI started performing at a higher level, too. I hadn’t been treating AI like a collaborative partner who needs the same foundational knowledge I’d give a person. I’d just kind of hoped AI would figure out my preferences and get it together eventually. I now give it the explicit context so AI can make smart, independent decisions. I built the context once. Now future AI tools and future colleagues can hit the ground running. Digging into "Your Brain on ChatGPT" Unknown — Wed, 10 Sep 2025 00:00:00 +0000 You’ve probably seen the headlines: “AI shortcuts are already making kids lazy“”“ChatGPT Use Linked to Cognitive Decline”“Using ChatGPT for work? It might make you stupid.” These stories reference a recent MIT study</a> titled “Your Brain on ChatGPT.” And these headlines, plus the way the study was framed, are the kind of sensationalized science communication that makes thoughtful conversations about AI and education harder to have. Yes, I worry about AI and education</h2> I’m big on AI. I work in enterprise AI and see how these tools can free people for more joyful, creative, high-impact work. I also have a young child, and I worry about what her education will look like when she and her peers have access to AI from day one. But this study doesn’t address those concerns. Instead, it amplifies fears with dramatic interpretations that go far beyond what the research actually shows. This didn’t study education</h2> Here’s what the MIT researchers did: Recruited 54 college students. </li> Paid them $100–150. </li> Asked them to respond to an essay prompt for 20 minutes with EEG caps measuring brain activity. </li> To get a “baseline” of thinking activity, participants spent two minutes doing rapid mental math, like multiplying 128 × 56 or adding 5,689 + 7,854 in their heads. </li> Then they wrote short essays responding to SAT-style prompts like: “Does true loyalty require unconditional support?” </li> “Can people have too much enthusiasm?” </li> </ul> </li> </ul> Some used ChatGPT, some used Google, some wrote without digital tools. The study took place over four months, with each participant completing three or four sessions over that time period. The paper claims, in the first paragraph of the abstract, to study “the cognitive cost of using an LLM in the educational context of writing an essay.” But what’s educational about this setup? No instruction. </li> No feedback. </li> No grades. </li> No goal. </li> Generic SAT prompts with no iteration or follow-through. </li> And you can’t move your head because it will mess up the signals. </li> </ul> When the session ended, students didn’t walk away with insight or skill. They walked to the sink with shampoo and a towel to wash EEG gel out of their hair. In classrooms, students have grades, improvement, feedback**.** Stakes shape engagement, as does actual interest in the topic. None of that was present here. This wasn’t education. It showed what happens when you introduce a tool to help with an otherwise tedious, one-off task. Study findings</h2> Here’s my summary of the key findings. And yes, if you’re coming from the belief that AI causes a cognitive deficit, these findings could look alarming: Students using AI showed different patterns of brain connectivity compared to those writing without assistance. </li> Their brains showed less coordinated activity across multiple regions. </li> They had trouble recalling quotes from essays they’d written minutes earlier. </li> Many reported feeling less ownership over their work. </li> By the third session, some students said ChatGPT felt “not reliable” for research or “not worth it” for certain assignments. </li> In the fourth session, sequence mattered: Students who started without AI and later used it showed increased brain activity. </li> Students who started with AI and then wrote alone showed continued reduced engagement. </li> </ul> </li> </ul> But to me, the reason we can’t extrapolate further is because this study felt much more like a tedious activity than an educational opportunity. It was more like giving students a worksheet of 20 long-division problems that they weren’t graded on and wouldn’t see again. If some used a calculator and some didn’t, would you expect the calculator group to remember the exact numbers they punched in? Probably not — they’d just do it and move on. Additionally, two teachers evaluated the essays and rated the ChatGPT-assisted essays lower. But as far I can tell, participants never saw those grades and never got feedback. And each session was a new prompt. Without iteration or direction, there was no chance or incentive for any real education. Researchers measured brain activity, then declared what it meant. Reduced connectivity = cognitive decline? Maybe. But it could also reflect efficiency, new strategies, or adaptation to a new tool. Moving forward</h2> I don’t take issue with doing exploratory lab studies like this or the methods used. My issues are the mismatch between the design and the claims about “education,” and the assumptions the researchers seemed to make. The abstract declares the study’s goal is to “demonstrate the pressing matter of a likely decrease in learning skills.” That doesn’t sound neutral or unbiased. Further research should add: Feedback, revision, and iteration </li> Real educational stakes </li> Projects students care about </li> A neutral starting point, not the assumption that AI is a “cognitive cost” </li> </ul> The MIT study really found that brains adapt when using new tools. That’s not surprising. Whether that adaptation helps or harms learning depends on design and context. Sensational headlines about “AI rotting our brains” don’t help us figure that out. I was ranting about this study to my husband and asked if he was equally upset. He replied: “No more than I am with all the AI companies promising AGI and superalignment by next quarter.” Fair point. The hype cycles run in both directions, and both deserve skepticism. Learn MCP with me, part 2: working with MCP servers and clients Unknown — Wed, 10 Sep 2025 00:00:00 +0000 In Part 1</a>, I looked into why MCP exists: it’s guardrails for LLMs to use APIs without screwing up. Next I wanted to dig into how to actually work with MCP servers. What got me started on this journey was discovering that Mintlify offers MCP servers</a> for all their hosted documentation sites. I saw this but didn’t really understand what it meant or how to use it. Can I curl an MCP server? How do I debug when things go wrong? What does the development workflow actually look like? I decided to explore using some of these Mintlify-hosted MCP servers to build a documentation research agent that will provide answers across several different tools I use. I went into this just trying to learn the technical details, but I walked away genuinely excited. Even though I knew theoretically what MCP should do, watching an agent I created intelligently decide “I need to search Mintlify docs AND Cursor docs for this comparison question” and then synthesize the results was pretty magical. Quick review: clients vs. servers</h2> To quickly recap: MCP server: exposes tools/resources that an AI can use. In this case, I’m using remote-hosted MCP servers that Mintlify sets up for docs sites. There’s a large list of other MCP servers here</a>. MCP client: the AI application that connects to and uses MCP servers (like Claude Desktop, Cursor, Windsurf, or one you build yourself). You configure the client to know about servers, then ask the AI to do things, and it figures out which tools to use. 1. Just get it working: plug into an AI app</h2> This is probably the best way to get moving. Here, I’ll use Cursor as an example, but the same approach works with Claude Desktop or other AI clients. I wanted to use remote servers to test this because I didn’t want to focus on setting up local servers. Setup for Cursor: Create an mcp.json</code> file in Cursor. This can be project-specific or global</a>. Cursor also has some one-click integrations listed</a> so you don’t even need to manually update a file. </li> Add the remote URL for any MCP servers </li> Ask the AI to do things </li> </ol> Here’s my ~/.cursor/mcp.json</code>, which connects to three different MCP servers that contain docs for the different tools I use (WRITER, Mintlify, and Cursor): { "mcpServers": { "Writer Docs": { "url": "https://dev.writer.com/mcp" }, "Mintlify": { "url": "https://www.mintlify.com/docs/mcp" }, "Cursor": { "url": "https://docs.cursor.com/mcp" } } } </code></pre> You can verify these URLs work by visiting them in your browser. They should return JSON describing the server capabilities. Try it out</h3> I was initially confused about what to do next. How do you test it? How do you know it’s working? It’s actually a lot simpler than I was making it in my head. Once you add your servers to Cursor, it’s just a regular chat. MCP servers are essentially just tools your AI can use. It’s like any other function you might give an AI, like one that calculates averages or fetches weather data. The magic happens in conversation: Ask: “What tools do you have available?” </li> The AI shows you what it can access across all your tool documentation </li> Ask: “How do I implement authentication across Writer, Mintlify, and Cursor?” </li> The AI searches each documentation server and synthesizes the results </li> Ask: “What are the API rate limits for each of these tools?” </li> The AI compares approaches across all three platforms </li> </ul> At this point, you might be satisfied. It works, it’s useful, and you don’t need to understand the details. But I was still trying to figure out what was actually happening under the hood. 2. What’s actually going on? Taking a look with curl</h2> My first instinct was: “Can I just curl this thing?” I was still confused about transport and protocols and what’s actually happening for the request/response pattern. So tried it out. Server inspection</h3> If you visit https://www.mintlify.com/docs/mcp</code> in a browser, you can see the server capabilities. It shows you the available tools, their descriptions, and parameter schemas, similar to an OpenAPI spec. But can I actually call the tools with familiar HTTP tools? Calling tools with curl</h3> This is where I learned MCP isn’t just REST with a different name. It uses JSON-RPC 2.0</a>, which requires a more structured approach. After some trial and error and checking out the MCP specification</a>, here’s what works: First, you need to initialize a connection and send an initialize request using jsonrpc 2.0</code>. This initialize method is from the MCP</a> spec, so it’s standard for all MCP requests. # 1. Initialize the connection curl -X POST "https://www.mintlify.com/docs/mcp" \ -H "Content-Type: application/json" \ -H "Accept: application/json, text/event-stream" \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "initialize", "params": { "protocolVersion": "1.0", "capabilities": {}, "clientInfo": {"name": "test-client", "version": "0.1"} } }' </code></pre> Success! It returns Server-Sent Events</a> with the actual available tools: event: message data: {"result":{"protocolVersion":"2025-06-18","capabilities":{"tools":{"SearchMintlify":{"name":"SearchMintlify","description":"Search across the Mintlify knowledge base to find relevant information...","method":"","pathTemplate":"","parameters":[],"inputSchema":{"type":"object","properties":{"query":{"type":"string","description":"A query to search the content with."}},"required":["query"]},"executionParameters":[],"securityRequirements":[],"operationId":"MintlifyDefaultSearch"},"listChanged":true}},"serverInfo":{"name":"Mintlify","version":"1.0.0"}},"jsonrpc":"2.0","id":1} </code></pre> Then, you can call a tool. You can see in the response above that there’s a tool called SearchMintlify</code> that takes a required query argument. The tools/call</code> method is specifically defined within the MCP spec as well</a>. The LLM can see from this tools list what it needs to plug in to provide when it calls the tool. # 2. Call a tool (note the exact tool name from the capabilities response) curl -X POST "https://www.mintlify.com/docs/mcp" \ -H "Content-Type: application/json" \ -H "Accept: application/json, text/event-stream" \ -d '{ "jsonrpc": "2.0", "id": 2, "method": "tools/call", "params": { "name": "SearchMintlify", "arguments": {"query": "API authentication"} } }' </code></pre> And when it works, you get back: event: message data: {"result":{"content":[{"type":"text","text":"Title: Authentication\nLink: https://www.mintlify.com/docs/api-playground/mdx/authentication\nContent: You can set authentication parameters to let users use their real API keys..."}]}} </code></pre> Here’s what the full flow looks like:</h3> This was really helpful for me to understand what was part of the MCP spec and what’s happening when an agent uses an MCP tool. I wouldn’t use curl when implementing an MCP client, but it’s useful if you want to: Verify server availability and capabilities </li> Test the initialization handshake </li> Debug tool calls and see exact response formats </li> Understand authentication requirements </li> </ul> To actually build a client, you’d want a client library. 3. Building it myself: an intelligent docs support agent</h2> Now that I understand the protocol better, I wanted to try to build my own agent that could work with these MCP servers. Using Amazon Strands with built-in MCP support</h3> I chose to use Amazon Strands</a> for this part, because I like the interface, it has integrations with WRITER</a> and lots of other model providers, and it happens to have MCP support built in. With Strands, you can swap out WRITER models for other model providers if you use something else, like Anthropic or OpenAI. To follow along, first you’d install Strands with Writer (or your model provider, you can find the info in the Strands docs</a>) and tool support: pip install 'strands-agents[writer]' strands-agents-tools </code></pre> You’d also need an API key for the model provider you’re using. Below I’ll walk through the agent I built to answer questions about the three different tools I’m using by pulling from their docs MCP servers. It creates connections to multiple MCP servers, combines all their tools, and lets the AI agent decide which ones to use for each question. See the full code in this Gist.</a> First, import the dependencies. import os from mcp.client.streamable_http import streamablehttp_client from strands import Agent from strands.models.writer import WriterModel from strands.tools.mcp import MCPClient </code></pre> The next section configures the Strands Agent. It creates streamable http connections to the three servers, configures the model to use Palmyra X5</a>, and stores the system prompt for the agent. class StrandsResearchAgent: def __init__(self): """Initialize MCP clients for documentation servers""" # Create MCP clients for each documentation server self.writer_client = MCPClient( lambda: streamablehttp_client("https://dev.writer.com/mcp") ) self.mintlify_client = MCPClient( lambda: streamablehttp_client("https://www.mintlify.com/docs/mcp") ) self.cursor_client = MCPClient( lambda: streamablehttp_client("https://docs.cursor.com/mcp") ) # Set up Palmyra X5 model self.model = WriterModel( client_args={"api_key": os.getenv("WRITER_API_KEY")}, model_id="palmyra-x5", temperature=0.3, # Lower temperature for more focused responses ) # System prompt self.system_prompt = """ You are a research assistant with access to documentation from Writer, Mintlify, and Cursor. When answering questions: 1. Determine which documentation sources are most relevant to the question 2. Search those sources strategically (don't search everything unless necessary) 3. Synthesize information from multiple sources when helpful 4. Provide practical, actionable answers with examples when possible Be efficient with tool usage - only search the documentation that's relevant to the specific question. """ </code></pre> The script then starts the chat loop. It opens the connections to the MCP servers so they’re available during the context of the agent chat. Then it provides the list of all the available tools from the MCP clients to the agent, along with the model and system prompt defined earlier. def chat_loop(self): """Interactive chat loop for research questions""" print("\n🤖 Strands Research Agent (Palmyra X5) ready!") print("Ask me about Writer, Mintlify, or Cursor documentation.") print("Type 'quit' to exit.\n") # Keep MCP clients connected for the entire chat session to maintain conversation history with self.mintlify_client, self.cursor_client, self.writer_client: # Combine tools from all servers all_tools = ( self.writer_client.list_tools_sync() + self.mintlify_client.list_tools_sync() + self.cursor_client.list_tools_sync() ) # Create a Strands agent with Palmyra X5 and all MCP tools # This agent will maintain conversation history across all questions agent = Agent( model=self.model, # Palmyra X5 model defined in __init__ tools=all_tools, # Open MCP tool connections defined above system_prompt=self.system_prompt, # System prompt defined in __init__ ) </code></pre> This is the part that really got me excited. I just pass the connections to the Strands agent as tools and it figures out the rest. I don’t have to write any logic for “when should you search Writer docs vs Cursor docs” - the AI agent sees the available tools and their descriptions and makes intelligent decisions about which ones to use. The last part is running the chat loop, where the script accepts input from the user, passes it back to the model, and streams the final results back to stdout. # Use the same agent for all questions to maintain conversation history while True: question = input("You: ").strip() if question.lower() in ["quit", "exit"]: break if question: print("🔍 Researching...") try: # By default, the agent streams the response to stdout # More info: https://strandsagents.com/latest/documentation/docs/user-guide/concepts/streaming/callback-handlers/ agent(question) print("\n") except Exception as e: print(f"❌ Error: {e}\n") # Usage examples def main(): agent = StrandsResearchAgent() # Start interactive chat agent.chat_loop() if __name__ == "__main__": main() </code></pre> See the full code here.</a> Here’s an example of using it: I ask: “What are the Mintlify and Cursor API rate limits?” </li> Palmyra X5 analyzes: Understands this is about API development and rate limiting </li> Smart tool selection: Decides to search Cursor and Mintlify docs for API rate limiting info </li> Targeted searches: Makes specific queries like “API rate limiting” </li> Intelligent synthesis: Agent combines results and provides practical guidance </li> </ol> What’s so cool is that I never told it explicitly “search both Mintlify and Cursor for this question“ or told it anything more than where to find the servers. It figured out from the question that it needed information from both sources, searched them independently, and synthesized the results. The bottom line</h2> MCP is still early, and the debugging experience reflects that. You’ll hit rough edges that don’t exist with traditional APIs. But when it works, it’s genuinely useful, especially for adaptive AI workflows where you want the LLM to discover and orchestrate tools on its own. It’s funny how quickly I want from “why do I even want MCP?” to “what? You don’t have an MCP server for that??”. Side notes: auth and debugging</h2> Auth</h3> One thing you might notice is that some MCP tools require authentication to external APIs. With MCP, authentication is just another fill-in-the blank; the MCP server tells the client which credentials it needs, and the client provides them for the server to then handle the actual auth. Interactive clients like Cursor or Claude can prompt you for API keys or other credentials when needed. If you’re building your own programmatic agent that needs credentials, you’d need to provide them upfront. You can handle this by passing authentication headers when creating the MCP client: self.writer_client = MCPClient( lambda: streamablehttp_client("https://dev.writer.com/mcp", headers={"Authorization": f"Bearer {writer_api_key}"})) </code></pre> This is one area where the MCP ecosystem is actively evolving. The MCP specification now includes comprehensive OAuth 2.1 support for HTTP-based transports, with proper authorization flows, dynamic client registration, and standardized discovery mechanisms. However, the practical reality varies significantly between different implementations. Debugging</h3> When things go wrong (and they will), you don’t have as many options for debugging as you typically would when working with traditional APIs, especially if the MCP server is hosted remotely. Available now: curl for protocol-level debugging </li> MCP Inspector</a> for local development </li> Server capability inspection via HTTP </li> Client-side logs (if your client exposes them) </li> </ul> The community is working on better tooling like MCP DevTools, protocol analyzers, and VS Code extensions that are being developed for MCP debugging. The ecosystem is moving fast, so hopefully the debugging experience will improve significantly over the next few months. I’m really excited about the new tooling around observability being added to the overall AI ecosystem. Mintlify has been very forward-thinking about AI integration, with MCP servers and several other AI-first features. Instead of treating AI integration as an afterthought, they’ve made it a core feature. It’s particularly useful when you’re building agents that need to work across multiple tools and platforms, like the research agent we built above.</li> </ol> Learn MCP with me, part 1: what is this even for? Unknown — Mon, 08 Sep 2025 00:00:00 +0000 I wrote this post because I found myself struggling with a very specific question: Why MCP when we already have APIs and so many ways to communicate programmatically? TL;DR: MCP isn’t replacing APIs; it’s a guardrail layer that makes LLMs less terrible at using them. Why MCP when we already have APIs?</h2> I come from an API development background: I’m comfortable working with, writing, and documenting APIs. Since MCP came out, I’ve been side-eyeing it like, “mhmm, yes I get it, it’s just another API.” But when I started to dig in, I couldn’t get past the question: what problem does this solve that REST + OpenAPI don’t already cover? We’ve already solved “connect different systems” multiple times. We’ve got REST, GraphQL, gRPC, even SOAP, and so many others. And then MCP shows up in 2024 saying “Here’s a standardized way for AI to connect to external tools.” What problem is it solving that REST + OpenAPI doesn’t already solve? It felt like the XKCD comic</a>. The answer: LLMs suck at using APIs</h2> After reading more, here’s what I found: LLMs are terrible API clients. They hallucinate parameter names, mishandle nested responses, and struggle with authentication patterns. Chatting with an LLM can sometimes feel so magical that I forget they’re all just probabilistic token generators. They’re not reading the OpenAPI spec or docs and using logic and reasoning to determine the correct API response. They’re coming up with the most common patterns they’ve seen in training, and sometimes those aren’t actually correct. How MCP fixes this</h2> MCP doesn’t replace APIs. It inserts a competency layer between LLMs and APIs. Here’s what it adds: Standardized tool interface: LLM just fills in parameters, no guessing endpoint names or auth headers. </li> Clean, structured responses: Instead of wrestling with 50-field JSON objects, the LLM gets simplified outputs. </li> Authentication abstraction: Tokens, OAuth, headers are all handled server-side, so the LLM doesn’t need to deal with any of that, it just needs to be handed the correct keys. </li> Discovery built-in: The LLM can ask “What tools are available?” rather than needing a preloaded spec. </li> </ul> Instead of fragile API calls, the LLM gets a structured template it just fills in. It gets information back in a reliable, standardized way. It doesn’t have to guess that a GitHub API needs repository_name</code> vs. repo_name</code>; MCP ensures it always gets the right parameter structure. Then you can plug that into agents or tools like Cursor or Claude so they can interact with those systems. Follow up: I’m still confused</h3> I still had a few questions. Mainly, if an LLM can’t just look at an OpenAPI spec and get it right, why would it magically succeed with MCP? The key is that MCP doesn’t rely on the model parsing and reasoning over a giant spec. Instead, the MCP runtime hands the LLM a structured template for each tool, enforces the parameters, and validates the output. The model isn’t “learning MCP” from training. It’s just filling in blanks inside a schema the runtime controls. That shift from “read this 50-page manual and hope you guess correctly” to “fill out this form, we’ll handle the rest” is what makes MCP reliably work where OpenAPI falls apart. When you would want to use MCP</h2> One of the biggest errors I’ve seen with people using AI is overengineering a solution. Sometimes it’s going to be a lot more reliable and straightforward to just integrate with one API than add a bunch of MCP servers. That said, there are definitely cases when MCP is the right choice: AI agents that adapt: a code review assistant that chooses tools based on changed files. </li> Chat interfaces with tool access: asking “Any errors in the auth module?” and having the LLM query Sentry + GitHub automatically. </li> Cross-system workflows: coordinating multiple tools without manually programming the details for each one. </li> </ul> The cross-systems workflows use case is where it gets exciting: the ability to coordinate across multiple systems without hardcoding every single step yourself. Imagine an AI assistant helping with customer support triage. It could: • Pull recent Zendesk tickets that mention a specific issue • Cross-reference related documentation in Confluence or issues in Jira • Post a suggested reply draft in Slack for a support agent to review With APIs, you’d have to write and maintain that integration flow. With MCP, the agent can discover tools at runtime and decide which to call, in what order. It’s less “pre-baked script” and more “AI reasoning with a toolkit.” That’s MCP’s sweet spot: enabling adaptive orchestration across services. When you don’t need MCP</h2> There are many cases when MCP and even agentic AI is overkill: You’re building deterministic workflows (cron jobs, fixed reports) </li> You’re writing code yourself (you don’t need a middleman) </li> It’s a single integration (REST is simpler) </li> You need full debugging control (direct API calls are transparent) </li> </ul> The bottom line</h2> MCP isn’t a revolution. It’s an evolution. It wraps existing API practices in guardrails so LLMs can use them without faceplanting. But, it’s still pretty exciting when you start working with it and seeing how quickly you can add integrations into an AI app. Now that I’m convinced MCP is worthwhile, I’ll dig into how to actually connect to MCP servers. That’s coming soon. We all fall down: notes from an ordinary outage Unknown — Sat, 06 Sep 2025 00:00:00 +0000 I love incidents. They’re like mini soap operas, tech stacks buckling under ordinary circumstances for unpredictable reasons, cascading failures, humans rushing to figure out what happened. So normal and each time so unexpected. It’s been a while since I worked in SRE and have gotten to see a good incident up close. So when a friend who works in infra at a streaming company told me about one particular outage, I wanted to dig into the details and learn more. A regular video processing job kicked off and suddenly the whole cluster was stuck for five days. It’s the kind of story that reminds me how the internet actually works: not as a gleaming futuristic system, but as a patchwork quilt of duct tape, YAML, and unfinished TODOs. Here’s my retelling, my best effort as an outsider through persistent questions. Sidebar: what even is Kubernetes?</h2> This story’s main character is Kubernetes. I’ve had the great fortune to not have to deal with Kubernetes (a.k.a “k8s”) directly. For those like me, here’s my best take on it: it’s like a manager for your apps. Instead of one person running one program on one computer, Kubernetes spreads workloads across many computers, makes sure things restart if they crash, and helps scale up when traffic spikes. It’s kind of like an orchestra conductor making sure dozens of instruments stay in sync, except the instruments are apps, and they sometimes set themselves on fire. How it started</h2> One of the teams at my friend’s company submitted a large video processing job for routine batch encoding work. These kinds of jobs fan out into tens of thousands of pods: little containers that encode video segments in parallel. There was nothing different about this job except that it was a bit bigger than usual. But then things jammed. The first clue came when their API gateway started refusing requests with mysterious error codes. Behind the scenes, Kubernetes was taking longer than 60 seconds to answer any kind of question, even one like “hey, what pods are running right now?” That delay alone was enough to break things, because their cloud provider gives up commands after 60 seconds. So from the outside as a user, the whole system looked frozen. Someone would try to kick off a new job, notice it wasn’t running, try to query what was going on, and nothing would happen. Sidebar: pods in plain English</h2> Pods are the smallest “work unit” in Kubernetes. Imagine every task you want to run, like “encode this 30-second clip,” gets its own little container. That’s a pod. They’re disposable, lightweight, and meant to come and go constantly. Pods help things run in parallel and keep discrete workloads isolated. Slack pings and silent dashboards</h2> This part of the infra team doesn’t rely heavily on alerts. Most of their workloads are batch encoding jobs, which should retry and recover automatically. The first notice of something wrong came in Slack with messages like: “Hey, the encoding job isn’t finishing…” </li> “Is the processing cluster down?” </li> “The batch queue seems stuck” </li> </ul> So the infra team went to the dashboards. But, nothing looked wrong. The metrics that lied</h2> Everything seemed fine when they went to look at the metrics. That’s when they remembered that their monitoring system, Prometheus, was configured to “assume everything’s fine” for 30 minutes if it stopped hearing from a service. If something died, Prometheus just kept reporting its last known state for 30 minutes, rather than showing that the service had entirely stopped sending metrics. To the dashboard, it looked like video encoding was proceeding normally. To the humans, it meant they had no actual insight into what or where things were breaking as the batch processing queue backed up. This was a known issue with their monitoring system, but it turns out that the engineer who started fixing it had left for another job, and the half-complete system stayed behind, quietly misleading everyone. The missing gatekeeper</h2> Eventually, after lots of digging, they discovered the real culprit: Kyverno, the “gatekeeper” service that approves or rejects new pods, had vanished. It hadn’t crashed. It’s not that it just had to spin up again. It had been evicted entirely. Why? Because another service from the security team had been given higher priority than even critical system services. So when resources got tight, Kubernetes politely kicked Kyverno out to make room. In this case, the security team hadn’t coordinated with the infra team when they set that priority. Infra assumed it was just another workload, not something that could outrank the cluster’s most essential services. Imagine a bouncer who has to check every person before they can enter the club getting kicked out when the club gets busy by someone shouting “FOR SECURITY REASONS.” No Kyverno means no new pods, including Kyverno itself. The long slog</h2> Part of the reason this outage dragged on was that Kyverno itself hadn’t been set up with any safety nets. Only one copy was running instead of multiple replicas because no one had updated the default setting from replicas: 1</code>. If there had been replicas, there could have been another copy of Kyverno available to let in the evicted one. And, there were no rules in place to prevent it from being kicked out. So when Kubernetes needed to free up space and Kyverno went away, with it went the entire cluster’s ability to process encoding jobs. Not having done Kubernetes work myself, I wondered why they couldn’t just delete the problematic jobs that started this mess. Well, apparently the jobs weren’t just sitting in the queue. They were part of Kubernetes’ Job and Pod system, which is designed to retry until success. If you delete one, the Job controller notices: “Oh, that pod disappeared, but the job isn’t done yet” and it immediately schedules a new one. And in this outage, the admission controller was down, which meant nothing new could start anyway, so trying to “clean up” jobs didn’t help. They’d just boomerang back or stay pending, still clogging the scheduler’s brain. The recovery</h2> The only way out was drastic: delete the admission rules that required Kyverno in the first place. That needed elevated permissions, so another team stepped in to rip out the safety checks temporarily. Once Kyverno was no longer required, the cluster slowly brought itself back online: The setup tasks could run. </li> Nodes became ready. </li> Kyverno was allowed back. </li> </ul> They restored the admission rules, this time with better safeguards: multiple replicas, disruption budgets, and higher priority. The encoding pipeline limped back to life, and the batch processing queue finally started clearing. Sidebar: why removing admission rules is a big deal</h2> Deleting admission rules didn’t sound that dramatic to me – just get rid of a few filters to get things back to normal. But it’s closer to removing the turnstiles at a subway station. Normally, those gates check every passenger’s ticket, which slows things down slightly, but ensures everyone who enters is allowed. Without them, trains move faster at first, but freeloaders and chaos slip in. That’s what happened here: taking down the rules let the cluster recover, but it also meant anything (good or bad) could sneak through until order was restored. Lessons</h2> This whole saga was a reminder that: Small defaults matter. Forgetting to set “run 3 copies of this critical service” can take down a whole processing pipeline. </li> Priorities collide. One team’s well-intentioned security tweak can accidentally evict the very thing keeping the system alive. </li> Observability isn’t optional. Dashboards that lie are worse than dashboards that scream. </li> Complex systems fail in loops. Once you get stuck in a cycle (“can’t start A without B, can’t start B without A”), recovery often requires outside help. </li> </ul> In the end</h2> What I loved about this outage was that it had less to do with machines failing and more to do with humans being human. Someone left a job and left their half-finished monitoring fix behind. Someone else set a priority without checking what it might evict. Someone forgot to configure replicas. All perfectly reasonable decisions in isolation. It was a reminder that our digital world runs on millions of these small choices, most of them invisible until suddenly brought under the microscope of a blameless postmortem. The miracle isn’t that things break. It’s that they hold together as often as they do. Most of the time, our duct tape and YAML and forgotten TODOs somehow keep the lights on. Until they don’t. When they’re not actively happening to me and I didn’t cause them by dropping a column in the production database.</li> </ol>

Code execution with MCP: separating context from computation

Set up your Anthropic API key</h1>
cp .env.example .env</p>

Edit .env and add your ANTHROPIC_API_KEY</h1>

Run the traditional tool calling example</h1>
python examples/traditional_tool_calling.py</p>

Generate MCP tool wrappers for the code execution example</h1>
python generate_wrappers.py</p>

Run the code execution example</h1>
python examples/code_execution.py`The examples use Anthropic’s Claude Haiku 4 model with the Anthropic SDK, but the patterns work with any model provider.</p>

Next steps</h2>

Read the Anthropic post on code execution</a></p> </li>

Learn best practices for writing tools for agents</a></p> </li>

Check out Cloudflare’s Code Mode explanation</a></p> </li>

Learn about MCP</a></p> </li> </ul>

On agency, abstraction, and AI-assisted coding

Testing docs IA with AI agents

Can AI help test IA? A Wikipedia game experiment

The Exploratorium boulder: why enthusiasm isn't always contagious

Docs metrics and the stories we tell ourselves

How onboarding a human made my AI smarter

Digging into "Your Brain on ChatGPT"

Learn MCP with me, part 2: working with MCP servers and clients

3. Building it myself: an intelligent docs support agent</h2>
Now that I understand the protocol better, I wanted to try to build my own agent that could work with these MCP servers.</p>

Side notes: auth and debugging</h2>

Learn MCP with me, part 1: what is this even for?

We all fall down: notes from an ordinary outage

Sarah Deaton

Making All Caps as a Service

The git commands I avoided for nine years

Costuming Claude Code

Claude Code beyond prompting

Sarah Deaton

Making All Caps as a Service

The git commands I avoided for nine years

Costuming Claude Code

Choosing a character</h2> Before settling on Moira, I tried on a few other personas, like:</p> Bob Ross (gentle, encouraging, every bug a happy little accident)</li> Werner Herzog (what would code be without a monster lurking in the dark? It would be like sleep without dreams</em>)</li>

Claude Code beyond prompting

Code execution with MCP: separating context from computation

Production safety: guardrails for code execution</h2> Giving an LLM the ability to execute code is powerful, but requires guardrails. Three key practices:</p>

Set up your Anthropic API key</h1> cp .env.example .env</p>

Edit .env and add your ANTHROPIC_API_KEY</h1>

Run the traditional tool calling example</h1> python examples/traditional_tool_calling.py</p>

Generate MCP tool wrappers for the code execution example</h1> python generate_wrappers.py</p>

Run the code execution example</h1> python examples/code_execution.py`The examples use Anthropic’s Claude Haiku 4 model with the Anthropic SDK, but the patterns work with any model provider.</p>

Next steps</h2> Read the Anthropic post on code execution</a></p> </li> Learn best practices for writing tools for agents</a></p> </li> Check out Cloudflare’s Code Mode explanation</a></p> </li> Learn about MCP</a></p> </li> </ul>

On agency, abstraction, and AI-assisted coding

Testing docs IA with AI agents

Can AI help test IA? A Wikipedia game experiment

The Exploratorium boulder: why enthusiasm isn't always contagious

Docs metrics and the stories we tell ourselves

How onboarding a human made my AI smarter

What I did</h2> Here are the pieces I put in place that made a difference:</p> Style, voice, and tone</h3> The biggest lift was that I finally wrote down all the style rules I’d been carrying in my head.</p>

Style, voice, and tone</h3> The biggest lift was that I finally wrote down all the style rules I’d been carrying in my head.</p>

Digging into "Your Brain on ChatGPT"

Learn MCP with me, part 2: working with MCP servers and clients

Here’s what the full flow looks like:</h3> </p> This was really helpful for me to understand what was part of the MCP spec and what’s happening when an agent uses an MCP tool. I wouldn’t use curl when implementing an MCP client, but it’s useful if you want to:</p>

3. Building it myself: an intelligent docs support agent</h2> Now that I understand the protocol better, I wanted to try to build my own agent that could work with these MCP servers.</p>

Side notes: auth and debugging</h2>

Debugging</h3> When things go wrong (and they will), you don’t have as many options for debugging as you typically would when working with traditional APIs, especially if the MCP server is hosted remotely.</p> Available now:</strong></p>

Learn MCP with me, part 1: what is this even for?

We all fall down: notes from an ordinary outage

Set up your Anthropic API key</h1>
cp .env.example .env</p>

Run the traditional tool calling example</h1>
python examples/traditional_tool_calling.py</p>

Generate MCP tool wrappers for the code execution example</h1>
python generate_wrappers.py</p>

Run the code execution example</h1>
python examples/code_execution.py`The examples use Anthropic’s Claude Haiku 4 model with the Anthropic SDK, but the patterns work with any model provider.</p>

Next steps</h2>

Read the Anthropic post on code execution</a></p> </li>

Learn best practices for writing tools for agents</a></p> </li>

Check out Cloudflare’s Code Mode explanation</a></p> </li>

Learn about MCP</a></p> </li> </ul>

3. Building it myself: an intelligent docs support agent</h2>
Now that I understand the protocol better, I wanted to try to build my own agent that could work with these MCP servers.</p>