Kintsugi

Inspiration

I run Aroppo, a creator-opportunity aggregator that depends on about nineteen scraping connectors. In one two-week window they made 744 fetches across 380 ingestion cycles, against sites that redesign whenever they feel like it. When one does, a selector dies, extraction quietly drops to zero, and I lose an afternoon reading HTML. Self-healing connectors were already on my roadmap. Then I read the Taskmaster brief (a complete workflow that takes action and sends the right information to the right place) and it was basically a description of the thing I needed. Self-healing scrapers do exist commercially, so I'm not claiming the idea. What I'm claiming is that this one is built carefully enough to leave running.

What it does

Patchwright watches a parser. When a source breaks, it asks Gemini what changed, rewrites the parser, proves the rewrite in an isolated sandbox against ground truth, has a second model family write a review, and hands the result to a human to approve. Code that hasn't verified never gets as far as the approval card. Code nobody approved never touches a live parser.

The live demo walks you through one repair. On the left is Fauxpost, a fake job-listings site I built for this, and a scraper that reads its twelve listings correctly. Press "Break the source" and the site's HTML changes the way a real redesign would: the class names the scraper relies on get renamed, tags get swapped, dates change format, titles get split across elements. The scraper now returns nothing, or worse, wrong data.

Then the agent takes over, and you watch each stage complete in the middle column. It notices the breakage, asks Gemini what changed on the page, has Gemini rewrite the scraper, and runs the new code in an isolated sandbox to check that it recovers all twelve listings with every field correct. When it does, the right column shows you the proposed code next to the old code, a table of which fields came back right, and a short plain-English review written by a second AI model (Gemma) saying what changed and what got riskier. Nothing has shipped yet. You press Approve, and it does.

If the agent can't produce a version that passes the check, it stops and says so, in a calm amber notice rather than a red error. I think that's the most important screen in the project. A scraper that always claims success is one that quietly corrupts your data.

How we built it

Gemini 3.7 Flash runs the diagnosis and patch agents through Google's ADK for JavaScript. The loop is explicit orchestration with ADK LlmAgents as the steps, because it retries conditionally on a deterministic verifier and a straight sequential agent can't express that.

Gemma 4 (31B) is a separate reviewer agent that writes the brief for the approver. It's a different model family from the one that wrote the patch, and the approve endpoint never reads its output.

It runs on Cloud Run as a single instance on purpose, with the reasoning written into the architecture doc. Firestore holds session state that survives a revision replace, Secret Manager holds the key, and a $100 budget alarm sits under all of it.

The ground truth can't lie. The mock site renders 12 records through a template, and a "break" rewrites only the template, so verification is always the same question: does the parsed output equal the records the page was rendered from? The bar is 12 of 12 items with every field correct.

Model-written code runs behind two layers: a subprocess with no inherited environment, and inside it a node:vm context stripped down to an allowlist of ES intrinsics. The server is Node stdlib, the front end is Vite and React, there are 141 offline tests, and a spec-first design doc records every decision and why I made it.

Disclosure: patchwright is new work built during the hackathon. It incorporates four small pieces from Aroppo (an ADK wrapper, a parsing idiom, design tokens and a build convention), itemised in DISCLOSURE.md in the repo. Aroppo production was read-only throughout.

Challenges we ran into

Most of what went wrong only showed up when I ran things for real. The docs described how the agent framework and the models were supposed to behave; the live behaviour differed in small ways that looked like bugs in my code until they weren't. Twice an agent did something strange for a reason that took an hour to find and a minute to fix.

Sandboxing code that an AI just wrote is harder than it sounds. My first version had gaps I only found by attacking it myself, and closing them changed the design: the sandbox is now two layers, because neither one was enough alone.

Production found its own set. Logging, concurrency, and a second AI model's behavior under strict output rules all surfaced problems that a hundred-plus passing tests could not see, because the tests never touched the real services.

Accomplishments that we're proud of

A 62-repair sweep across the full mutation space came back 62 of 62 on Gemini 3.7 Flash, including 17 of 17 on the tier that had been running at 78% on 3.5. I tuned nothing afterwards, because tuning against zero observed failures just fits noise. The sandbox held against every escape attempt I threw at it, including the Function-constructor and prototype-chain tricks. The whole build, spikes and sweeps included, cost about five dollars in Gemini spend. And the retry loop has already recovered a hard break live on the hosted URL in under twenty seconds, which I did not expect the first time I saw it.

What we learned

Invariant ground truth meant verification couldn't be gamed, even by accident, and that mattered more than any prompt I wrote. Counting rows told me nothing about the mutation that leaves twelve items at 100% coverage with every title wrong; only per-field value fingerprints caught it. Switching models did more than prompt tuning would have: a head-to-head on identical seeds cut wall time by more than half and removed a whole failure class. And the failures weren't where I expected. Diagnosis was right every time. Code generation was what slipped.

What's next for patchwright

Point it at Aroppo's real connectors, starting where the agent proposes and a person ships, and let individual connectors earn auto-merge as their track record grows. Add a syntax-repair retry, since that's the one failure class I've actually observed. Move the runtime tier out of process memory so it can scale past one instance; the path for that is in the architecture doc. And keep the reviewer. A second model family at the approval gate is the part of this I'd least want to give up.

Built With

• gemini