Journal by Eric.https://ehfeng.com/https://ehfeng.com/dear-reader/https://ehfeng.com/dear-reader/Sat, 01 Aug 2026 00:00:00 GMT I clicked merge again this morning and felt a bit numb. It is the late agentic summer, and leaf has subsided to leaf. I don't build anymore; the agents do. It started with the code. But hey, whatever. Code was barely meant to be written or read by humans anyways. We'd already been abstracting away the machine with JITs and GCs and transpilers. What's yet-another-abstraction? Then it crept into the issues, PR descriptions, code reviews. And is anyone even _reading_ these scrolls of text? In the last week, the humans at Kernel "authored" (but really just output) 274k words in PR descriptions and comments. More words than all five seasons of Breaking Bad. In the face of this reckless verbosity, what choice is there but "Have your agent call my agent"? Where once I was steering, now i-have-adhd and the agents are steering me. I find myself nodding along with the super-intelligence and vanishing deeper into the sea of tokens. I look back and wonder, where am I, Eric, in all these words? I am here, in the typos before the autocorrect, in the pauses between paragraphs, in the undo history of my rewrites. I am here, hoping you might be nodding along and maybe write back.https://ehfeng.com/your-users-table-is-chaos/https://ehfeng.com/your-users-table-is-chaos/Tue, 26 May 2026 00:00:00 GMTSelf-serve means your users table is chaos. When anyone can sign up with any email address, the users table is inevitably polluted with throwaway emails. Self-serve users to orgs is a many-to-many relationship, so this means your orgs table is also chaos: users will create throwaway orgs to test out your product. Chaos is fine. Ever grep through `ps`? It's a mess down there. You mostly care about order for your internal tooling: your CRM, support queue, financial reports: your "human-in-the-loop" tools. ### Filtering for paid It's worth filtering out free signups, both users and orgs. They are the main source of chaos. You get spammers, abusers, etc. Most PLG companies have a "cheap" tier (ours is $30/month) and opting into the sales funnel or triggering the free trials (with CC) is typically dead simple. Free users should mostly be addressed via passive education: docs, product onboarding, blog post, etc. ### Domains Domain(s) map to org, account, or company. While some domains will exist across multiple orgs, you can usually decide on “primary” org for each domain. We should have domain “tombstones” to mark cases where a domain should not be included in that org. In case where an org has multiple domains, the newest domain should be considered the primary domain. Track this with a unique `domain` to org id to SaaS id table. Picking the primary org should have a few heuristics: - Verified domains with automatic invitations - Paid tier (the higher, the more primary) - Recency as a tie breaker ### Emails Email(s) map to user, contact, or people. While *emails* are enforced as unique in Kernel, users might be part of multiple orgs. Like domain, there is *always* a primary org for business purposes and mapping to that is sufficient. Track this with a unique `email` to user id to SaaS id table. ### Tombstones There are always exceptions to every rule. Alerting on cases where a domain or user could be associated with multiple orgs (where both orgs are legit) and creating “tombstones” to disassociate a domain-org or user-org connection. Both domain and email tables should have a “tombstone” array type column, each within a GIN index for fast membership querying. ### Consolidation Domain/email drift can happen in two directions: SaaS added a new contact or Kernel has added a new user. When a new contact is added in the SaaS app, we should attempt to find its matching account. If a new user is added in Kernel, we should attempt to link it to existing SaaS contacts. If we fail to find a link, no-op. ### Deletion If a user membership is removed from an org, recalculate and reassign. If a user is deleted, we simply leave those contacts as unassigned to an account. In Pylon, deleting a contact will delete underlying issues. ### Webhooks We need triggers from our GTM tools, billing, and product (user, org, or membership updates). You probably also want a once-per-minute cron to force updates. ### Dry-run Dry-runs are obviously required for all integration code, but I'm especially interested in how to maintain these integrations over time with agents. I have no interest of manually maintaining my mini-Fivetran. Most of it was vibe-coded with a large spec on the data flows and critical business decisions. Another engineer starts work on it this week, so it'll be interesting to see how we collaborate on the code.https://ehfeng.com/gtm-in-the-loop/https://ehfeng.com/gtm-in-the-loop/Fri, 22 May 2026 00:00:00 GMTThe primary job of go-to-market is communication with humans outside the company. Sales, marketing, support—all of it is outbound communication. And communication has voice: tone, positioning, the way you frame a problem before you pitch the solution. That voice should be, over time, trainable. I've been training an ["eric voice" skill](https://gist.github.com/ehfeng/c03c426e3087789b3ff5bc6a57be2783) for Claude Code. It drafts emails, Slack messages, blog posts, and customer replies in *my* voice. Not a generic "professional tone"—my actual patterns, my tendency to lead with the technical detail before the business implication, my habit of short declarative sentences. It's not just a good imitation. It's often *better* than me—more consistent, more disciplined about structure, less prone to the rambling I do when I'm tired. The first draft is almost always tighter than what I'd write from scratch. But agents converge toward the mean. Left unsupervised, any LLM drifts to median internet prose. The human in the loop isn't there to write—they're there to *steer*. To notice when the positioning has drifted, when the tone has gotten too safe, when the framing no longer matches what you learned from the last ten customer conversations. This is the reverse centaur: the human isn't doing the work, but the human's judgment is what keeps the work from being mediocre. This is a lot of how kings of old used to hold court. Listening to petitioners wasn't just good for the petitioners—it was how the king stayed calibrated on what the people actually needed. GTM leadership works the same way. You're not just communicating outward. Every sales call, every support ticket, every marketing response is a signal coming *in*. The CEO who delegates all GTM to agents without listening loses the signal. The one who stays in the loop—reviewing drafts, steering tone, reading the inbound—keeps both the communication and the company's positioning sharp. ## Headcount as infrastructure Think about headcount the way you'd think about infrastructure design. App servers are the easiest to replicate—stateless, horizontally scalable, throw more instances at the problem. The database is the hard part, because contention and consistency are delicate to scale. The CEO is the primary database. They hold the state that everything else reads from: the company's positioning, the product vision, the voice, the priorities. The CEO is the juncture between GTM and EPD—the single source of truth that both sides replicate from. On the GTM side, you need two read replicas. GTM is embarrassingly parallel—more outbound, more coverage, more conversations—but it all has to be consistent with the positioning the CEO sets. Contractors and agents slot in naturally here. The replicas execute; the CEO reviews the output and course-corrects. GTM in a loop. On the EPD side, you also need two read replicas, but for a different reason. Engineering has real contention—two engineers touching the same system create merge conflicts, architectural disagreements, subtle bugs from misaligned mental models. The codebase needs a coherent design philosophy, not five different patterns for the same problem. You need the replicas not for throughput but for incident coverage and redundancy—the system can't go down when one node is unavailable. In theory, you could compress further: three people. One GTM DRI, one EPD DRI, and the CEO as the read replica for both sides. The CEO doesn't need to be the primary on either—they just need to stay consistent with both, which is exactly what a read replica does. The DRIs own their domains; the CEO maintains coherence across them. At three people, replication lag is essentially zero—you're all in the same room, same Slack, same context. Compare that with a 100-person company, where the CEO is weeks behind on what engineering shipped and sales is pitching features that don't exist yet. The lag isn't a people problem; it's a scaling problem. Three people with agents can stay in sync in a way that a hundred people with meetings cannot. ## The back of the house vanishes For companies of roughly five people, the "back of the house" tends to disappear entirely. Recruiting, HR, finance, procurement, office management, legal, IT, analytics—these functions either get absorbed by the founders or handled by agents and services. This isn't new. Five-person startups never had an HR department. What's new is that the *ceiling* for operating without these functions has risen dramatically. Agents handle procurement workflows, generate financial reports, manage IT provisioning, draft legal documents for human review. The point at which you *need* a dedicated person for any of these has moved from ~20 employees to maybe never, for a certain class of company. The org chart of a five-person AI-native company looks less like a small version of a big company and more like a core team with an orbit of agents, each trained on a different function. GTM in a loop. Support in a loop. Finance in a loop. The human is always in the loop—but the loop is the unit of organization, not the department.https://ehfeng.com/cross-company-issue-threading/https://ehfeng.com/cross-company-issue-threading/Wed, 06 May 2026 00:00:00 GMTLinear "Customer Requests" are a first step. We need more. Linear customer requests helps us better present problems to product, but it does not handle customer validation flows. ## Yesterday This is what I've seen in the past: 1. Customer reports bug. 1. Support aggregates, hands off to product. 1. Product fixes bug. ## Today This is what we're doing at Kernel. I'm calling the customer-facing team "Integrations" in this context. 1. Customer reports bug. 1. Integrations\* repros, aggregates, hands off to engineering. 1. Engineering fixes bug. 1. Integrations tests with customer, verifies or re-escalates. ## Problem SaaS or IaaS adoption always requires a customer champion: someone who believes in your service and is willing to do the implementation work. Ownership of that relationship gets muddied over time as multiple teams compete for *champion time*: - Infrastructure: incidents - Sales: upsell or renewal - Marketing: customer stories, co-marketing, cross-selling - Product: new releases - Support: bug fixes However, champions are *your company's most valuable people*, even more than your own employees! You can hire new employees; if you lose a champion, it's almost impossible to find a new one in that company. This is why sales teams are _so_ protective of their relationships: partly out of selfish interest, but mostly because they understand the *relationship is valuable*. But this isn't a communication problem, it's a *prioritization* problem — which resolves itself if you have a sorted list. Effectively, you need a *per-customer* issue tracker. I'd love to use Linear for this, but two problems: 1. Login requires a *Linear account*, which we cannot guarantee our customers have. 2. Linear has no cross-account issue dependencies, and we absolutely do not want 200+ customer teams in our Linear account. GitHub remains the default. We already maintain a per-customer GitHub repo[^1], so it's natural to use the issue tracker inside it. The thread we want: a customer files an issue in *their* repo, we mirror it to internal Linear, link the two with a "blocked by" relationship, and when Linear closes, the customer issue updates with the resolution. The customer sees a single thread; we see a workflow. ## Tomorrow Invite customers to a GitHub repo. Collaborate on issues *and* code. Agents on both sides of a vendor relationship are siloed today: our agent reads our Linear; the customer's agent reads their tracker. They are gated _by human communciation_ (email, Slack). The shared repo collapses that — both companies' agents read the same threads, post fixes to the same PRs, inherit the same code context. The repo *is* the integration spec, the bug tracker, and the deploy target. [^1]: When working with customers on their integration, we've found it helpful to start tracking the code artifacts in a repo, giving both the humans and the agents valuable context. Also, sharing arbitrary code on email is often viewed as suspicious by email providers and hard to read in Slack. Because Kernel is an infrastructure provider, it'd be nice (note to future self) to have a "deploy to Kernel" or "run in Kernel" for Playwright or Kernel app invocations.https://ehfeng.com/continuous-customer-acceptance-testing/https://ehfeng.com/continuous-customer-acceptance-testing/Wed, 29 Apr 2026 00:00:00 GMTFor customer contracts, legalese is worse than code. Code is more precise, consistent, and enforceable. Just like we test functions with unit tests and APIs with integration tests, customers should define acceptance tests — not uptime percentages — for vendors they rely on. Software has leaned on vendors for decades: every `npm install` is effectively adding a new vendor. Yet in package management, [dependency hell](https://en.wikipedia.org/wiki/Dependency_hell) is managed by version pinning and testing. The frustrating part of _hosted_ vendors is the impossibility of version pinning and the futility of integration testing (unless specifically offered by the vendor, like [Stripe Sandboxes](https://docs.stripe.com/sandboxes)). Continuous deployment has meant continuous _change_, but when you build on infrastructure, continuous instability. For Kernel, browser infrastructure adds another layer of instability: the target websites themselves. The DOM, captchas, memory consumption — all changing constantly, and impossible to sandbox[^1]. When an automation fails, it's a three-way blame game: _your code, the browser infra, or the target website_? A workflow automating a UnitedHealthcare site might break because the site shipped a new version, because Kernel's stealth proxies started failing, or because the customer's Playwright script has a bug. It's one thing to know your automations might break. It's another to find out because your _customer_ is reporting it. [^1]: Unlike Stripe Sandboxes, which work because Stripe controls both sides of the API, Kernel can't offer a "Kernel Sandbox." Browser automations run on the real web — that's the whole point. A browser with a proxy that blocked all POST, PUT, and DELETE requests would likely _break_ websites, even if the automation script itself was meant to be idempotent. The typical response is SLAs and escalations — but escalations mean you're _waiting for things to break_. Then the back-and-forth inherently _slows things down_: isolating the problem, writing the fix, deploying the fix, validating the fix. Each human step takes hours. And each time the fix _fails to resolve the issue_ increases frustration. Customers should define their own acceptance tests as part of their contract. Run them post-deploy — no individual customer should be able to block deploys — but alert both the customer and Kernel when an automation might fail. Then, when engineering begins to work on a fix, they just have to _make the test pass_. Test-driven development, applied to customer workflows[^2]. [^2]: This would need to be limited to enterprise contracts, but it's akin to "a dedicated engineer" or "uptime SLA" that many infra companies already provide — just codified in acceptance tests instead of legalese.https://ehfeng.com/dialogue-format/https://ehfeng.com/dialogue-format/Sun, 19 Apr 2026 00:00:00 GMTWhat does it means to "publish" a blog post anymore? Once published, a post represents what I thought on that date. The back catalog is stale. But I keep musing and updating my thinking. Open source business models, growth engineering, customer engineering---these are conversations I've been having for years, with different people, and yet my posts remain frozen. Fine, if you think of a blog as a log, but what is that like for a new person (or agent) getting acquainted with me? I wanted two things. First, the ability to *resume* a topic. Not write a new post that vaguely references an old one, but actually pick up where I left off, with the latest version always being my current thinking. Second, I wanted the *process* visible. Not just what I think now, but how I got here---which conversations changed my mind, which prompts from Claude sharpened an argument, which coffee chat made me rethink a paragraph. The [writing dialogue](/writing/human-excellence-after-ai/) on my recent posts already showed the prompt-by-prompt drafting process with Claude. But those were stored as giant YAML files alongside each post---a parallel data structure that duplicated the content and had to be maintained separately. It worked, but it was brittle and manual. Then I realized: git already tracks exactly what I want. Every edit to a post is a commit. The commit message can carry metadata---who I was talking to, what prompted the change, what type of input it was. The diff between commits *is* the evolution. I was maintaining a second system to track something my first system already did. ## How it works A dialogue is just a markdown file with a git history. Each commit that changes the file is an entry. The commit message carries structured metadata as [git trailers](https://git-scm.com/docs/git-interpret-trailers): ``` I think taste and legibility are one and the same actually taste is judged by other humans. legibility is convincing other humans that you have taste. With: Claude ``` The `With:` trailer names who or what precipitated the change---could be `Claude`, could be `Sarah Chen <sarah@example.com>`, could be `Paul Graham`. Commits without a `With:` trailer still show up in the timeline, but only as dates---silent edits with visible diffs. ## The build pipeline At build time, a [script](https://github.com/ehfeng/ehfeng.github.io/blob/master/scripts/generate-dialogues.mjs) walks `git log` for each post file, parses the trailers, extracts the content at each commit, and writes a JSON file. Astro reads that JSON to render the dialogue viewer---the side panel on posts and the full `/writing/{slug}/` pages. The script caches against the latest commit hash per file, so `npm run dev` stays fast on repeat starts. In CI, the GitHub Actions workflow checks out with full history (`fetch-depth: 0`) and generates fresh. A `prepare-commit-msg` git hook pre-fills the trailer template when I commit a post file, so I don't have to remember the format. ## Why not a database? Why not YAML? Why git? Because the data *is* the history. Every version control system already stores exactly this---who changed what, when, and why. Adding a database or sidecar file to track changes to a file that's *already tracked by git* is a category error. The commit message is the natural place for metadata about a change. The diff is the natural representation of what changed. I was building a shadow of a system I already had. The tradeoff is commit message discipline. Every meaningful edit needs a well-written message with the metadata trailers. That's a real cost. But it's also a forcing function---it makes me think about *why* I'm changing something, not just *what* I'm changing. The git log becomes a publication artifact, not just a development tool. ## A post is the beginning of a conversation Grok on X must cost a fortune to run. It's also easily the most valuable feature that platform has ever shipped---every tweet becomes a starting point, and you can pull a real back-and-forth into public view by just asking. Dialogues should be similar. A post is still a post: a coherent *unit* of thought. But the path to it is continuous. You evolve. Which probably means post URLs are commit-tagged---`/customer-engineering` is just the latest, the way `main` is a named pointer that auto-advances. Permalinks live at the SHA. And then what *is* a blog post? Mostly a soliloquy---an internal conversation made public. What makes it a dialogue is the (hopefully tasteful) addition of AI. I think I actually *missed* on the first cut. I focused on the AI chat itself, which is honestly not that interesting, for the same reason most *podcasts* aren't: they're not well thought out. What I want: direct editing, but on save AI immediately gives feedback on your *changes* and on the *coherency* of the post. It adds to the dialogue on the side, and its notes *highlight* the parts of the post they're about. You chat back, voice or text. Publishing updates the post *and* summarizes the conversation behind the change. One of the more boring things about blogs is the lack of public discourse. Comments don't fix it---why craft a beautiful thought in someone else's comment section, to be read by 1% of an already-small readership? Better to link out to your public platform, where the conversation is happening anyway.https://ehfeng.com/human-excellence-after-ai/https://ehfeng.com/human-excellence-after-ai/Sat, 18 Apr 2026 00:00:00 GMT[John Henry](https://en.wikipedia.org/wiki/John_Henry_(folklore)) has been in the SF zeitgeist lately. He won the race against the steam drill, and then it didn't matter. The drill replaced him and everyone like him. Physical strength used to define worth. Machines changed that. Strength didn't disappear---it just stopped being *economically* meaningful. Now it's something cultivated for health, for aesthetics, for personal satisfaction. Intelligence is in the middle of the same transition. Working tirelessly on complex mental tasks requiring knowledge and experience---that's what I was valued for, by myself and by every company I've worked at. Agents can do it now, cheaper and faster. What remains? Sure, *virtues* still remain. Temperance, loyalty, forgiveness, etc. But what might *mark* excellence in our new future? Imagination? Even when AI can answer any question perfectly, can tutor you in any skill, or generate breakthrough scientific data, we must still bridge the natural world to *our* understanding. Einstein generated no new experimental data, yet his [*Gedankenexperimente*](https://en.wikipedia.org/wiki/Einstein%27s_thought_experiments)—imagining himself riding a beam of light, watching a clock tower recede—gave us the metaphors and theories to help us understand our physical reality. Conviction! Conviction stands in opposition to AI. AI is a non-deterministic trend towards the mean; conviction is the inner mettle to stand apart. The truths we hold in opposition to accepted wisdom or the cruelties of circumstance. Look to [Deng Xiaoping](https://en.wikipedia.org/wiki/Deng_Xiaoping): purged twice during the Cultural Revolution, his son paralyzed by Red Guards. He returned each time with the same beliefs, led China's reform era, then retired in 1989. Three years later, at 87, with no title and no obligation, he got on a train and embarked on the [Southern Tour](https://en.wikipedia.org/wiki/Deng_Xiaoping%27s_southern_tour)---pushing China towards markets when the entire party apparatus, terrified by the Soviet collapse, wanted to retreat. Taste. It is conviction made legible. After lying dormant in one person's heart, taste finds the form to express itself---with words, fabric, code, or light. [Tadao Ando](https://en.wikipedia.org/wiki/Tadao_Ando), a boxer turned self-taught architect, built the [Church of the Light](https://en.wikipedia.org/wiki/Church_of_the_Light): a concrete box with a cruciform slit in the wall. It's beautiful, but its legibility comes from our need for respite from an overstimulated existence. Always, kindness. Paul Graham [observed](https://paulgraham.com/mean.html) that mean people fail---that meanness makes you stupid, locks you into zero-sum thinking. Kindness is a type of fragile, exposed conviction---to treat another not based on how they treated you today, but to connect to who they are and why they hurt. You cannot stop their pain, but you can offer them a chance to accept and forgive their own hurt. It's that fragility that makes it so special, that agents don't risk. A type of soft, subtle bravery that helps us matter to one another, when our strength and intelligence no longer do. Imagination, conviction, taste, and kindness. Maybe letting go of intelligence, with all its wry sarcasm and towering superiority, won't be so bad. :)https://ehfeng.com/software-as-a-shim/https://ehfeng.com/software-as-a-shim/Sat, 18 Apr 2026 00:00:00 GMTI'm writing this post in a dialogue with Claude. I'm not just querying, coding, or automating. I'm drafting, refining, and even rewriting my thoughts via a dialogue with AI. I'm no longer looking for some kind of "perfect writing app." My thinking environment is now a conversation. What does that mean for SaaS and software broadly? ## SaaS is becoming a shim Software companies sell tools. Tools presuppose a human operator. If the operator is an agent, the tool is an implementation detail. Take Figma. Figma's brilliance was making design collaborative and browser-native. But if an agent can go from product spec to production-ready design in minutes---and Anthropic's Claude Design suggests this is months away---then Figma's value proposition changes. You don't need multiplayer cursors if there's no human designer collaborating. The agent doesn't need a GUI. It needs an API, or more likely, it just writes code directly. The same logic applies across the stack. Project management tools presuppose humans coordinating work. Analytics dashboards presuppose humans interpreting data. CRMs presuppose humans managing relationships. If agents do the underlying work, the tool layer compresses into a thin shim between the agent and the outcome. Adobe is the starker example. Photoshop, Illustrator, Premiere---these are instruments. Instruments are only valuable if someone is playing them. If an agent can produce the same output without the instrument, the instrument becomes a legacy cost center. Adobe has been bolting AI features onto Creative Cloud, but that's adding a motor to a horse. The horse is the problem. This doesn't mean every SaaS company dies tomorrow. But the value of the software layer is collapsing. What used to be a product is becoming a feature of the AI platform. ## What survives What agents *can't* replicate is more interesting than what they can. ### Physical infrastructure Cloudflare has edge nodes in 300+ cities. AWS has data centers with custom silicon. Twilio has contracts with carriers in 180 countries. You can't vibe-code a point of presence. These companies become utilities---low-margin, high-volume, essential. They survive, but they stop being "tech companies" in the venture sense and start looking like power companies. ### Human networks and the data they generate YouTube has billions of hours of video. Instagram has billions of images. Google Maps has billions of user-contributed corrections. LinkedIn has the professional graph. Airbnb has the hosts. The moat is the same in every case: billions of humans using the product day-in and day-out, generating data that no agent can replicate from scratch. An agent can *use* this data but can't *generate* it. These platforms face a strange future though. If agents are doing the transacting---booking the Airbnb, searching the listings, watching the videos---then the platforms become agent-to-agent marketplaces with human assets underneath. The software layer becomes trivial. The value is the humans and the data they've already produced. ### Regulatory capture Not regulatory *licenses*---agents can generate applications at massive scale. What matters is regulatory *capture*. In the past, a dozen banks per state applied for licenses in any given year. When agents can generate applications, that scale multiplies faster than the government can adapt. Governments will inherently react---through action or inaction---by favoring incumbents. The banks, insurers, and broker-dealers that already hold the licenses will be protected not by the difficulty of applying, but by the bottleneck of a bureaucracy that can't process a thousand applications where it used to process ten. ### Taste This is the most interesting survivor. Linear will endure because Karri Saarinen has a *vision of how software teams should work* and Linear is the embodiment of that vision. Their refusal to add custom fields for years wasn't a product gap---it was an argument that custom fields are where processes go to metastasize. That's not UX polish. It's *ideology encoded in software*. An agent can replicate Linear's UI in a weekend. It cannot replicate the conviction about what should and shouldn't exist. Software-as-opinion---Linear, Basecamp, Notion---is durable because customers are paying for someone to *tell them no*, to shape their thinking, not just execute their commands. Humans will pay a lot for taste---or at least the perception of it. Louis Vuitton and Ferrari are incredibly valuable companies despite their products being vestigial. But no one has a monopoly on taste. Linear will be worth billions, but never trillions. Most SaaS products don't have taste anyway. They have feature checklists. They're undifferentiated CRUD apps with a billing page. Those are the ones that evaporate. ## Margin compression If agents can replicate any SaaS product's functionality, margins compress to the *risk premium* of building it yourself. There's always some margin---building it yourself carries legal and financial risk, and agents, being non-deterministic, can't guarantee 100% uptime. But those margins might collapse from 80% to 50% or even 20%. The winners survive as massive "lambda-as-a-company" machines: a few skilled humans making taste and strategy decisions, with agents doing everything else. Value accrues from breadth of usage, not thickness of margin. Software becomes the rotisserie chicken---sold near cost to keep customers in the ecosystem. The companies with moats in atoms (infrastructure, carrier relationships, regulatory capture) or in accumulated human data will weather this. Pure software companies whose value is "we wrote code that does X" will not. ## Human compression It's not just software that compresses. It's the humans inside companies. When each person has 100 agents behind them, a 10,000-person workforce becomes 100. "Replaced" is the wrong word for what happens to any individual---their productivity gets multiplied until fewer individuals are needed. But from the perspective of employment, the distinction is academic. HR---the *managing masses of humans as resources* function---disappears. When all that remain are stakeholders and agents, you don't need an org chart. You need a liability structure and a shared taste for what you're building. Could 100 people manage Facebook's properties? Probably. What changes is the composition: a small core of humans with exceptional judgment about where to point the agents, and an army of agents executing. ## What's left When anything can be built, the only question is whether what you're building is worth building. That was always the question, but tools were slow enough that most of us never had to face it. We could hide behind the difficulty of execution. Agents remove that place to hide. What remains is taste---the willingness to have a point of view about what should exist and what shouldn't. That's not a software problem. It's a [human one](/human-excellence-after-ai/).https://ehfeng.com/customer-support-in-the-age-of-coding-agents/https://ehfeng.com/customer-support-in-the-age-of-coding-agents/Thu, 09 Apr 2026 00:00:00 GMTCoding agents aren’t just writing more software. They’re *generating* software like games have procedurally generated terrain. This has changed the support *demand*, so how must support operations evolve? ## Moving targets Vibe coding and plastic software have arrived. Agents mean code stability and software stability have universally declined. We're shipping faster and code *improves* faster, but we're constantly in a state of things being broken *at the edges*—not only in our own codebases, but in our customers' and our vendors'. Stability is sacrificed because every business feels a few months away from irrelevance. Historically, stability and support were hand-in-hand. Ubuntu had “LTS” releases because support was *tied to stability*. How can you ask your support *team* to keep up with the latest code changes? New bugs appear at the edges, and as soon as a bug is understood, an agent fixes it and deploys in minutes—but that fix may introduce new edge cases of its own. Support teams must track both—the breakages *and* the fixes. The broad feature might look the same, but *how it works and therefore how it breaks* is constantly changing under the hood. I've literally had conversations where I have to *verify when* a customer hit a bug, because we may have landed a fix since then. ## Interventions, not questions Customer support used to be answering questions around “opaque interfaces”. Kernel has SDKs and APIs. I can help you *understand* them, but they're fixed points. People would ask “How do I set up X?” or “How do you bill for Y?” This old way of thinking produces knowledge bases you can search over: docs, StackOverflow, FAQs. And because those interfaces were stable, the knowledge bases were *relatively* up-to-date. That's gone. Questions about our interfaces have been soaked up by agents. When setting up Kernel, customers are asking their agents, and those agents are poring over our public docs and repos. Most of the time, customers don't even need to ask because the agents have done the setup work. Support has shifted to managing interventions. Almost every ticket is now “This feature isn't working, tell me why.” Customers ask for server logs and expect code fixes. They *push into our code*, [literally even contributing](https://github.com/kernel/kernel-images/commit/b5aeca9) to our open source repos. It makes sense—asking for a code change used to be a major request, weeks or months. Coding agents make software *feel* like it should be responsive, fluid to our needs. Behind every bug, a customer thinks “I could vibe-code a fix in 5 minutes.” ## Humans in the loop Unstable software plus complex requests means support must shift too. Customer work is splitting into two types: relationship management and intervention triage. Relationship management will exist for the same reason you still talk to a sales associate to buy a Rolex. Humans trust humans, especially for major purchases. It's also an *embarrassingly parallel problem*—two account managers, two sets of assigned customers. The relationships and egos are complex, but it's a tale as old as time. Agents will augment this world without fundamentally changing it. The best hotel concierges can send up towels at the press of a button, but the art of hospitality remains. If even [a full AI-pilled tech founder](https://x.com/zeeg/status/2041900723211006228) wants a human, the human touch has no expiration date. Intervention triage is where the change is happening. In the last month, I've *changed*. I've stopped looking at Kernel docs, and I no longer train new support staff on them. Everything comes from conversation, relationships, and code. Each conversation, I open a new terminal session in a directory with a rapidly-evolving CLAUDE.md, all of Kernel's repos downloaded locally, and a directory of skills for diving into logs, databases, events. The CLAUDE.md outlines how to tackle issues and—importantly—gives me an "internal" explanation and drafts a basic customer response that gets copied to my clipboard via `pbcopy`. My favorite part is my addendum to the customer-response instructions: "Provide code samples." This *broadens the spectrum* of communication—I'm speaking to the human in English and to their compute in code. If a customer reports a bug, my agent can trivially write code to repro. If I can't repro, I send the customer *runnable code* and ask them to modify it into a repro case. Their agent can take my repro, compare it to their production code, and find the difference. Before, I only spoke to the human customer. Now I'm increasingly talking to their agent directly. “So what? You could always send code snippets.” The difference is scale and cost. Code is as cheap as prose now, and *scaling* these conversations is possible. When you're *collaborating* on code, support isn't a queue or even a conversation. It's Hanabi—the cooperative card game where you can see everyone's hand but your own. Support and customer are solving a problem together as fast as possible, but communications are constrained by policy and privacy. We're figuring out whether the next "play" is a code fix in Kernel or the customer's codebase. For long-term collaboration, you eventually want to create shared repositories with your customer to track what workarounds and samples have been passed to this customer in the past. After thousands of tickets, I can confidently say human will remain in the loop. The agent almost never one-shots a solution. Not because the models are bad—they're good at reading code, querying logs, drafting responses. But support issues that reach a human are *inherently hard*, because it is always a [moving target](#moving-targets). Therefore, support will inherently involve *steering*: recognizing when the agent is going down the wrong rabbit hole or just being lazy and nudging it in the right direction. This is why the logging matters more than any individual skill. I'm not fine-tuning models. I'm building a library of crystallized steering—"when you see X, check Y before Z"—loaded into the agent at the start of every session. A playbook written by *actual support conversations*, not guesses about what support might look like. The skills compound. The problems get harder. The steering gets more precise. ## Context for support agents Agentic support requires context. Before, context was the docs. Maybe a knowledge base. The problem is that these are just "knowledge caches" without a coherent revocation or TTL. Agents allow us to derive knowledge from the ground truth. So what is that new ground truth?  *Conversational*: what questions have already come up, what repro code has been sent, what tickets closed how, what was said in the latest sales call. To speed up access, I vibe-coded a simple MySQL cache. *Relationship* is the health of the customer—who they are, what they're trying to accomplish with Kernel, the current state of the relationship (happy, dissatisfied). It lives in your CRM, managed by the account owner. *Product* is which surface areas they use, which known bugs or feature scenarios they're likely to hit. For instance, Kernel has a `diagnose-aw-snap` skill for a class of browser crashes. Customer-facing and product-facing engineers both contribute to these skills, and a post-deploy agent step updates them against new commits. ## Agent to agent support Two *agents* talking across the customer/company boundary. The human's role becomes gating *trust*. Look at the workflow at the top. There are three gates—actual queues with actual humans at them, not three layers of verification: - A *prompt-injection* agent screens everything coming into `support@`. Flagged messages get blocklisted until a human reviews. The models are [good enough](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks) at this now that I don't lose sleep over it. The real risk isn't the jailbreak—it's data *export*. An attacker who can't access info or execute with privileges gives up. - *Privilege escalation* requires a human. Read-only on proprietary data doesn't. The moment the agent wants to write, run sample code against a customer's resources, or touch anything with export potential, a human signs off. Code can exfiltrate; that's the gate. - *Customer-facing output* gets human steering—every reply from `support@`, every escalation into #user-discuss, every draft Linear issue. The draft is usually close, but one-shotting is rare—the human tweaks the tone, adds context the agent missed, or redirects the response entirely when the diagnosis was off. OpenAI's [privacy filter](https://openai.com/index/introducing-openai-privacy-filter/) is another fascinating model for preventing data privacy breaches. This is what I'm focused on. I've evaluated other off-the-shelf vendors, but in infrastructure, truth comes from code and evaluating truth requires reading, writing, and executing code. Kernel runs its own unikernel infrastructure, which gives us isolation and (relatively) safe execution. Piggybacking on that for support agents is nearly free—and we already have the browser the agent needs to drive. Support used to be a knowledge problem: memorize the interfaces, maintain the docs, answer the questions. Coding agents have both made that world obsolete and open the doors for a new world. The value of support is no longer knowing how the software works—an agent can read the code faster than any human. The value is managing agent inputs and outputs to orchestrate company and customer code.https://ehfeng.com/what-is-customer-engineering/https://ehfeng.com/what-is-customer-engineering/Thu, 02 Apr 2026 00:00:00 GMTI've done engineering (Snowflake), sales & marketing (Numeracy), and a “hybrid” [growth engineer](https://ehfeng.com/what-is-growth-engineering/) (Sentry). At Kernel, I’m a customer engineer, a title I find myself often explaining, internal and external. So, time to write it down. You can define a function in three parts: - What is its purpose? - What is success/failure? - How does it work with other functions? ## What is the purpose? Customer engineering **scales customer obsession**. Customer obsession means listening to and delivering on customers' needs[^1]. Customers are the **ground truth**. They are how we touch grass. While no individual customer is “always right”, customer obsession course-corrects our product instincts. Founding teams are customer obsessed in the early days, but as the number of customers grows *exponentially,* customer obsession breaks down. Time for customers declines linearly as focus is split on recruiting, fundraising, partnerships, product roadmaps, and internal management. A founder’s ability to focus on specific customers, not just the broad customer base, declines, and has definitely disappeared by the Dunbar number (150) of customers. The last decade of product-led growth means self-serve customer bases scale faster, earlier. A decade ago, a public SaaS company might have a thousand customers. Now, you’ll get to a thousand before Series B, and that thousand might have all signed up in the last year. This volume creates the problem of **scaling** customer obsession very early on. ### Why not scale using the product? When customer obsession scaling breaks down, many companies shift to product obsession, because software scales (mostly). The dream is the product is self-documenting, customers are self-onboarding, and all your charts are up and to the right. You build a feature, update docs, and add it to onboarding. But most users are skipping the product onboarding or have trouble understanding the docs. They’re busy and they just want to create a browser and *move on*. Product obsession is right for the *product team*, but it’s not *enough* for the business as a whole. You’re asking customers to meet you at your product, instead of you meeting them where they are. Your early adopters will soak up everything you ship, but as your customer base expands and matures, you increasingly need to meet them. ## Scaling customer obsession Trite as it is, AI unlocks the ability to scale customer operations from an engineering perspective, instead of a business COGS problem. Customer obsession never scaled well before because customer support is overwhelmingly treated as a cost center, with the exception of companies like Zappos or Hilton, where it’s their brand. But those are fundamentally *service* companies with little to no core product of their own. At product companies, it’s hard to imagine customer support or even account managers ever getting paid more than the engineers. AI changes this. Not just because AI can reply to emails. The entire nature of customer communication has changed. Let’s talk through the lens of reactive & proactive, fixes & features. | | Reactive | Proactive | | --- | --- | --- | | Fixes | Bug fixes | Health checks | | Features | Feature requests | Feature recommendations | ### Reactive Reactive means responding to inbound: bug reports and feature requests. **Bug fixes** are the most straightforward quadrant. Either the customer misconfigured their code or our code’s behavior is incorrect. Since AI agents arrived, you don’t get the typical FAQ-level questions anymore: everyone is asking their agents, long before asking you. The questions I get are about bugs — “something unexpected happened, my agent can’t fix it. Can you see something I can’t?” And a lot of conversations turn into back and forth Claude screenshots. It’s not about having AI automate replies. The next step is letting customer and company AI’s have supervised conversations to solve not only customer problems, but diagnose and solve *product gaps*. In doing so, customer support blurs the line between COGS and R&D. For instance, when addressing issues, I no longer send just messages: I send code artifacts. AI enables “show don’t tell” at a support ticket level because writing sample apps or repro cases cost pennies instead of hours. **Feature requests** are the more interesting reactive quadrant. We collect them, bagged and tagged, until we’re confident this is the right direction for the product roadmap. Most times, these issues (with customer requests attached) should clearly aggregate *customer pain*, not propose *product solutions*. The fascinating part is *validation*: after release, we follow up with customers on whether this *solved their problem*. Founding teams do this naturally — every feature ships with a mental list of who asked for it and a Slack message checking in. But as the organization scales, validation gets lost. Historically, infrastructure customers were sticky, so even when they don’t adopt your latest features, they don’t churn. But that’s no longer the case. Swapping providers is one prompt away. Customer value is the *only* churn mitigation. Validating that features *solve their case* is just as essential as bug fixes. ### Proactive Companies love *doing* proactive comms, but how many of us truly enjoy them? For self-serve, you’d serve them a steady drip-drip of email campaigns and in-product tours. Except I don’t remember the last drip campaign I liked. I don’t remember the last product tour I didn’t mash the Escape button on. At the enterprise tier, you end up scheduling a monthly or quarterly check-in: often valuable but often slow-moving and difficult to scale. Proactive communications break down into two types: *health checks* for fixes and *feature recs*. Because proactive comms aren't requested by the customer, they must be managed more carefully. To ensure you're not just creating spam, each communication must be centered on *customer value*, which comes from two things: working products and working relationships. Working products means following up on all the dozens of little issues, making sure nothing gets dropped. Working relationships means human-coherent milestones: What is the customer trying to accomplish? What’s the champion’s career timeline? What are the blockers to each? **Health checks** are bug and performance audits. Look at a customer's usage, spot misconfigurations or silent failures, and reach out before they notice. Same work as reactive bug fixing: conversations should result in fixes on both sides, not just the customer. **Feature recs** are more interesting. When you ship a new feature, there should be a follow-up *per customer*: Open → In conversation → Evaluating → Adopting → Done, with Deferred or Declined as exits. These are customer engineering's issue statuses, but for adoption. How can you be proactive at scale, aka self-serve? Let me start with this: *don't drip*. You need to stop thinking about support as *tickets* and more as ongoing conversations. In a world where every shopping site has *product recs* and media is full of *product placement*, what if support had **personalized feature recs**? Every inbound support request is a natural transition to talk about your other product features, to drive adoption, to demonstrate and solve a problem **proactively**. Using Kernel Profiles and experiencing a bug? Let me fix that for you and also remind you that Managed Auth actually *solves the problem for you*. “While I have you on the line” becomes “while you have that kernel.js file open”. And these recs can be personalized not just to the customer’s use case, but to their language, stack, and toolset. A Python shop gets Python examples. A team on Terraform gets Terraform modules. What if you could send a prompt good enough to one-shot the adoption? Not a generic migration guide, but a concrete, ready-to-apply change tailored to their codebase. Support is uniquely empowered to shepherd the customer base forward. When things are *broken*, customers are most ready to *change* — they’re *activated*. **Products** inevitably have bugs, but responsive support and timely resolution builds trust in the **service**. And once that trust in the *service* is earned, you can use that same relationship to drive customers to adopt new features or even new products. ## What is success & failure? Infrastructure businesses are power-law. Stripe had Lyft, Shopify; Twilio had Uber, Airbnb. And yet a customer that is 20% of total revenue can produce the same amount of support load as a free customer. But also that same free *user* could test out a tool on weekend project and then bring it to work and end up as a massive account. PLG means self-serve doesn’t need to *convert* to be valuable. Marketing your product is *expensive* and the highest quality marketing is always word-of-mouth. ### Success Success is a customer fully utilizing the product. Whether or not a given customer has *volume* isn’t up to you. Whether a customer understands and adopts all the applicable *features* is. Metrics map back to the grid. *Reactive fixes*: response times and resolution rates. *Reactive features*: quality of the issue delivered to product engineering. *Proactive fixes* (health checks): customer happiness (easily judged by agents, not CSAT scores). *Proactive features* (recommendations): feature adoption. I trust that with each new feature or fix, we’re building something customers want. And by delivering that to the customers, the revenue will follow. Customer value and education have a long-term payoff, well beyond financial reporting cycles. The young couple spending a single night at your hotel deserves just as much attention as the traveling salesman, even if they spend a fraction. The value of a person *today* is a poor predictor for their *future value*, because great engineers curate the best tools and bring them wherever they go. ### Failure Churn is an *incident*, not a failure. Just like product engineering incidents, customer churn can be caused by our own bugs, competitive landscape, or even just customer fundraising cycles. We often see our Hobbyist churning because they don’t have enough usage. Customer engineering delivers churn postmortems, where each churn event, self-serve or enterprise, is understood as either a company failure or a customer journey (product or company shut down). *Failure* is customer disappointment. And we measure that in failed conversations and failed relationships. And that doesn’t just include support tickets that don’t resolve, but in all customer conversations, like proactive outreach, that result in customers getting frustrated with and losing trust in us, the customer team at Kernel. ## How does it work with other functions? There are two major functions that intersect heavily with customer engineering: sales and product engineering. ### Sales Customer engineering should not be making sales. Customer engineering is focused on delivering overwhelming value to the customer, regardless of whether they upgrade. Your greatest advocate is the engineer who switches companies (or starts their own) and *brings their tools with them*. And sales *much* prefers when customers are already sold on the product and they just need to *close the deal* (focusing on legal, compliance, finance roadblocks). Customer engineering should be handing off *insanely qualified leads* by driving customer success first and letting sales harvest the revenue later. ### Product Engineering The relationship between product and customer engineering is a question of **context management**. At any moment, all products have dozens of feature requests and hundreds of bugs. Trust me, I worked at Sentry. Building any individual feature or fixing any bugs doesn’t matter. Fixing bugs hit by hundreds or thousands of users doesn’t matter. *Addressing user pain matters* and that requires talking to users. The primary responsibility of customer engineering is to give product engineering enough context to start immediately. For feature requests, deliver a concrete customer use case. For bug fixes, a repro case. For both, a discrete list of customers that are or will be impacted. The threshold is **context**. Product engineering has the product context: stability, uptime, technical debt, design, etc. Customer engineering has customer context: for each customer, what are they using the product for, what features are they using, what is the state of the relationship, etc. When the fix does not require product *context* but requires a lot of customer context, customer engineering should address it. This happens most often with **bugs**, where documenting a corner case requires more effort than prompting a fix. When the fix requires understanding the product *system* as a whole, it’s always better to let product engineering’s code owner triage it. Most changes *should* end up in product engineering’s side, but the rapid turnaround enabled by customer engineering can save a customer relationship. I recently fixed a [Live View Safari bug](https://github.com/kernel/kernel-images/commit/6fda5ff4dd8ed3c59540fdda0a1347e9e23194c6). At first, I thought it was a minor bug: our live view seemed flaky at accepting user input for the first few seconds. But for this user, it was the difference between using Kernel or not because a significant portion of his audience (11%) uses Safari. On the other hand, our Live View VP8 was causing major CPU load on safari on Macs and iPads because they do not support hardware-level VP8 decoding. I completely lack the context on VP9 encoding. In this situation, customer engineering's job is to gather, aggregate, reproduce and triage customer feedback to hand-off to the part of product engineering with sufficient context. *Come for the product, stay for the service.* [^1]: Listening can be just as critical as delivering. Communication itself is valuable, just like during [incidents](http://ehfeng.com/incident-support/). Customers **feeling** heard and **knowing** your priorities/timelines can help them plan, even if it doesn’t change anything today.