Ted Tschopp

The Two Business Cases for AI Agents

2026-11-09T00:00:00-08:00

The email arrives with a photograph and a short note. “Same seal as the pump we bought in 2008. Can you get one by tomorrow?”

The nameplate in the photograph is missing two digits. There are three versions of the pump in the catalog. Someone at the parts counter has to figure out which one the customer owns before anyone can promise a delivery date.

Imagine you run that distributor. You could use an AI agent to help the quote desk deal with emails like this more cheaply. You could also sell a next-day sourcing service to customers who will pay to get a hard-to-find part located and verified. Both jobs start with the same email. They come to the investment meeting with different evidence. The new service needs paying customers. The quote desk needs a cost it can take out of the budget.

There is a third decision waiting for both teams. A model supplier will eventually ship another version. Some instructions and scripts that helped the old model may be work the new one can do. How do you find out what you can take away?

Sell the new service

The distributor already sells parts. The new thing is a promise that customers can buy. Send us what you know about the broken pump, and by the next business day we will give you a verified source and a lead time. A person can deliver the first version of this service with a phone, a catalog and a way to take payment.

That pilot tells you something a demonstration cannot. Will a maintenance manager actually pay $100 for the promise? It also gives the team the first version of the process. Record how many paid orders customers place and how many the desk can finish. For each request, keep the part you identified, who verified it and when the answer went out. Those records become the beginning of the data model and the first cases against which the team can test the agent.

Say the team gets 200 eligible requests in a typical month. In this example, it manually fulfills 30 paid orders at $100 each. At $60 in delivery cost per order, the service contributes $1,200 a month before shared overhead. Suppose a workflow with the model can handle 45 orders a month. It costs $40 to deliver each order, plus $200 a month to run the workflow. Getting it started costs $2,000.

Start with the 30 orders you already have. At the same $100 fee, revenue is $3,000 a month either way. Over twelve months, the lower delivery cost adds $2,800 in contribution after the launch cost. Now find 15 more orders a month from customers willing to pay for the same promise. At 45 paid orders, monthly revenue rises to $4,500 and monthly contribution to $2,500. Over those twelve months, the gain over the manual pilot rises to $13,600 after the launch cost.

You can test whether the desk has room for those extra orders before you know whether customers will place them. The sales case needs both. If buyers move over from an existing paid service, account for the sales and margin that move with them. If the objective is to attract outside capital, track funding commitments as a separate result.

Greenfield helps because you can design a small process around what people are willing to buy. You can change the intake form or narrow the promise without unwinding years of practice. You still need a way to say what good work looks like. In this example, the team asks for at least 98% correct results and a turnaround within one working day on 95% of cases. The new service can use the same old catalog, inventory and supplier records as the quote desk. How much infrastructure you inherit is a separate question from where the money comes from.

Take a cost out of the quote desk

Walk over to the existing desk and follow an email from arrival to the point when a correct quote reaches the customer. A written procedure helps. So do the people who can tell you what they do when the procedure runs out. One employee knows that a certain vendor reused part numbers. Another knows that a retired brand still appears on the plates people photograph. Some of that knowledge is in a catalog. A contractor also maintains a table of verified aliases and the script that searches it.

Here the advantage of a mature process is that you have something to compare. At 1,000 quotes a month, the desk spends $5,000, including $1,000 for the alias routine and its support contract. Look at ordinary cases, bad photographs, near matches and requests that should be sent to a person. Record correctness and elapsed time all the way through review. I’ve used cost per accepted outcome for this kind of comparison because the retries and corrections belong to the job. The Cost of a Finished Job

Faster work can give people time back. That time has value, especially when someone is waiting for an answer. A field study of a customer support assistant found a 15% average increase in issues resolved per hour. To claim a reduction in operating and maintenance expense, the budget owner at our parts desk also needs to show that the team can reduce overtime, end a support contract or take another identifiable expense out of the operation. Generative AI at Work

Give the model a particular job

I’d start with the part of the request that needs interpretation. A general-purpose language model can take the email and photograph, propose a part family and decide which approved lookup to try next. It can ask for the missing dimensions if the evidence runs out. A straightforward workflow can call the model once for a routine request. Some exceptions may justify letting the agent choose among several lookups.

The price comes from the transaction system. The catalog and manufacturer specifications tell you which parts are compatible. A person with the right authority decides what to do when two plausible parts remain. Put these boundaries in the tools and workflow so they apply each time the agent acts. When the model lacks an up-to-date fact, give it a lookup. When a rule is clear, implement it as a rule. A narrower machine learning model may be the right choice for a recurring classification task if you have examples and it performs better on them.

Today’s general-purpose model still struggles with some of those old vendor aliases. So keep the alias routine and give the agent access to it. Now suppose the whole desk costs $3,650 a month before the $1,000 contract. That’s $350 a month below the original desk. After a $2,000 cutover, the first year saves $2,200. The business case can stand on that result if the quality and turnaround also hold.

The table and the script do different jobs. The table holds old names that people have checked. The script looks them up. Before you switch off the script, put the verified aliases in the catalog and give someone responsibility for keeping them current. Give both workflows access to the same records. Keep compatibility, pricing and approval checks in the systems responsible for those decisions.

Run the upgrade with the routine turned off

Use the same cases for the model you have, the new model with the script, and the new model without it. The desk requires 98% correct quotes and a turnaround of two hours or less on 95% of cases, including time spent in review. It also requires separate passes on two kinds of request. When a vendor has reused a part number, the workflow has to verify the manufacturer and the compatible part. When two plausible matches remain, it has to send the case to a person before releasing a quote. A miss on either test can block the change even at 98.9% overall. The usual pricing and approval checks still apply.

Say the new model without the script comes in at 98.9% correct, with an hour and a half for 95% of cases, and passes the required tests. That earns a trial on live requests with people still releasing the quotes. You now have an answer to whether the upgrade can do the work without the script. You still have to decide whether to buy the upgrade.

From the original quote desk, suppose the total cutover would cost $3,000. At $3,800 a month for the new workflow, ending the $1,000 support contract would save $11,400 over twelve months after accounting for the cutover. Keep paying for support and the same comparison shows a $600 loss.

If today’s model is already running, start with the $4,650 the desk spends each month. The original cutover is behind you. Suppose another $1,000 is still needed to switch. End the support contract and you save $850 a month, or $9,200 over twelve months after the switch cost. Keep the contract and the new model costs $150 more each month. Including the switch cost, keeping today’s model saves $2,800 over those twelve months.

A different desk may still need the contractor. Keep that bill in the calculation until someone can cancel it. If the trial holds and the switch pays, roll it out with a fallback and retire the script once the desk can do without it.

There is precedent for removing instructions that newer models can do without. Cursor reported cutting roughly two-thirds of its system prompt and described using A/B tests to tune the system on live traffic. Model changes can go the other direction too. OpenAI rolled back a GPT‑4o update in ChatGPT after it made the model too agreeable. I would keep the production version fixed where the provider allows it and run a new release through the same workflow tests before moving traffic. Cursor’s engineering write-up · OpenAI’s account of the rollback · Model Portability Is Not AI Portability

Try the Agent Business Case Lab

Open the Agent Business Case Lab and start with the distributor. In Growth, set Paid orders placed per month to 30 and leave fulfillment capacity at 45. See how much of the case comes from lowering the cost of delivery. In Operating cost, turn off the option to cancel the support contract. The delivery checks can still pass for the upgrade without the script, even when keeping today’s model costs less. Set Unsafe near-match tests to Failed to see what happens to the upgrade without the routine. Choose Start blank to enter your own figures and identify where they came from.

If you’re choosing where to begin, pull up one real request with the people who have to finish it. Follow it until a customer has an answer. Somewhere between that blurry photograph and the correct seal, you’ll find a job you can price, a rule somebody has to own, and a case worth keeping for the next time the model changes.

Human Approval Has a Speed Limit

2026-11-05T23:00:00-08:00

Human Approval Has a Speed Limit

Before you increase what your agents can produce, decide how much your people can responsibly review.

Imagine a team receiving 120 requests an hour. An AI system handles the routine work and sends 20% of the requests to a person. Each human review takes five minutes.

That sounds manageable. The machine handles most of the work. People look at the exceptions.

Now count the minutes.

Every hour produces 24 reviews. At five minutes each, those reviews require 120 minutes of human attention. Two qualified people would have to spend the entire hour reviewing just to keep pace.

There is no time left for a meeting, an unusual case, a conversation with a customer, or correcting yesterday’s mistake. There is no time to help a colleague understand something the system got wrong.

You kept the human in the loop. You just don’t have enough humans to handle the volume of reviews.

This is the next practical question for organizations putting AI into their work: How much work can reach the review queue before the review stops being credible?

Count the hours people actually have

In How Much Work Can Your AI Safely Own?, I described review debt: faster assembly creates more work packages for experienced people to inspect. In The Cost of a Finished Job, I argued that human accountability requires enough context, time, expertise, and authority to make a real decision.

The next step is to turn those requirements into a limit on incoming work.

Return to our example. Over eight hours, the team receives 960 requests. The 20% needing review produces 192 cases. At five minutes each, that is 16 hours of review work.

Suppose there are two reviewers. Both have eight-hour working days. But after their other responsibilities, each can protect four hours for this queue.

The team has eight review hours available. Sixteen hours arrive.

By the end of the first workday, eight hours remain unfinished. Under the same assumptions, another eight hours join them the next day. After ten workdays, the queue contains 80 hours of review work, equivalent to 960 average cases.

These are illustrative numbers. The arithmetic is useful because it exposes a common planning mistake: counting a person as a full day of available capacity while assigning that person several other jobs.

A headcount tells you how many people belong to the team. A review plan needs to say when those people can do this particular work.

Review arrangement	Review demand per day	Protected capacity per day	What the averaged model shows
Two reviewers with four hours each	16 hours	8 hours	Eight hours added to the queue each day.
Two reviewers with eight hours each	16 hours	16 hours	No spare capacity. Any starting queue remains.
Three reviewers with eight hours each	16 hours	24 hours	Eight hours available for existing work or variation.

The eight-hour cases show the boundary in the calculation. They are not a recommendation to schedule uninterrupted reviewing for an entire working day.

The queue is part of the control

Recent accounts of deployed systems make the human decision point visible.

In its Nov. 2 account of building Copilot Cowork, Microsoft describes human review becoming a bottleneck as agents produced more code. The team used automated reviewers for early feedback and retained human checks for critical changes. It also reports a case in which several reviewer agents missed four blocking issues that people subsequently found. This is a first-party engineering account, not an independent comparison of review methods. It does, however, show why faster automated feedback and sufficient human judgment remain separate design questions.

An AWS and U Mobile account published Nov. 4 describes a different setting. Its telco helpdesk investigates a subscriber’s problem and prepares corrective actions. A support agent reviews and confirms before changes execute on the live account. That arrangement places a person at a consequential point in the workflow. The account does not establish how much review traffic that person can absorb.

The capacity question follows from both examples. What happens when requests arrive faster than qualified people can inspect them?

The system might wait. Someone might work late. A team might begin approving larger batches. Reviewers might spend less time recovering context or challenging the proposed action.

Those are different operating responses. Each deserves an explicit decision. A growing queue should make the limitation visible before people quietly absorb it through overtime or thinner review.

If approval is one of the controls that justifies letting an agent act, then the organization needs evidence that the control can operate at the authorized volume.

Give incoming work a limit

The basic calculation is small enough to keep beside the workflow:

Daily review demand = requests × fraction needing review × average review minutes.

Daily review capacity = qualified reviewers × protected review hours × 60.

Divide demand by capacity to see the load being offered to the team. In our example, demand is 200% of capacity. That does not mean the reviewers are performing at 200% utilization. It means twice as much work is arriving as their protected time can accommodate.

At 100%, the averaged model has no spare capacity. An empty queue can remain empty if every assumption holds. An existing queue cannot shrink. A longer case or an interruption has no unused time waiting for it.

A team can choose to plan below that boundary. The appropriate margin depends on its work: variation in case difficulty, the cost of delay, access to specialists, and the consequences of an inadequate decision.

Suppose, purely as a planning example, the team chooses to admit new work up to 80% of its protected capacity. Eight available review hours become a new-work budget of 6.4 hours, or 384 minutes.

Each incoming request creates one minute of expected review work: 20% multiplied by five minutes. The corresponding intake limit is therefore 384 requests per workday.

Keeping the original 960 requests would require five reviewers with four protected hours each to meet that example target. The team would then have 20 review hours, with 16 committed to new demand.

The 80% figure is an assumption to test, not a standard of safety. The useful change is making the relationship explicit. When protected review capacity changes, the amount of new work the system admits should be reconsidered too.

An agent platform can enforce this through a dispatch limit, a bounded queue, or a pause before starting additional work. The implementation depends on where work can safely wait. A customer request may still need to be accepted and acknowledged even when its processing must be delayed or escalated.

Someone should own that decision, including what happens when a time-sensitive case arrives at a full queue.

Returned work also needs attention

Our first calculation assumes each reviewed case needs one pass. Real work can have mistakes that require additional review passes. Additionally, a review might find something as complete, only to find out later that it was incomplete or incorrect, necessitating further review.

Consider the arrangement with two reviewers protecting eight hours each. New demand exactly fills their 16 hours. Now suppose 25% of reviewed cases need one additional four-minute pass.

The average workload becomes six minutes per reviewed case: five minutes initially, plus one minute for the additional pass averaged across all cases.

Daily demand rises from 16 hours to 19.2. Capacity remains at 16. The queue gains 3.2 hours each day.

This example includes exactly one additional pass. A workflow with repeated returns needs to measure those passes too. Counting only the first approval request would hide part of the job.

The same care applies to the work class. Five people are not interchangeable capacity if only one can judge a particular exception. Separate the specialist queue before pooling the hours. Otherwise, the total can look comfortable while the cases that matter wait for the same person.

Review time also has content. A useful review may require opening the source record, checking the relevant policy, resolving an ambiguity, and understanding what the proposed action will change. Measure that work, including corrections and follow-up. The time taken to click an approval button is a poor substitute.

Know the process before you accelerate it

If this calculation and way of thinking feels strange to you, you are not yet ready to put AI into this process. You need to understand the work well enough to plan for what happens when more work arrives.

That may be uncomfortable to hear, especially if someone is already asking you for an AI implementation date. You may have inherited a process whose measurements were never collected. The people doing the work may have kept it moving through experience, informal conversations, and extra hours. Give yourself and those people time to make that work visible.

Start by following a request from arrival to a finished outcome. Sit with the people who handle it. Ask what they check, where they wait, what comes back, and which decisions require someone with particular experience or authority. Include the awkward cases that do not fit the procedure.

Then collect the information needed to make a capacity plan:

How much work arrives, over what period, and how sharply arrivals vary.
Which requests need human review, why they need it, and who can perform it.
How much active effort a complete review takes, including gathering context and checking evidence.
How often work returns and how much additional effort each return requires.
How many qualified review hours are actually protected after other responsibilities.
How much work is already waiting, how old it is, and what happens when a decision is late.

Use existing records where they are useful. Observe or sample the work where the records are incomplete. Include busy periods and difficult cases. Keep active effort separate from elapsed waiting time; both matter, but they answer different planning questions.

Your first estimates will have gaps. Mark them, ask the people closest to the work to challenge them, and replace them with observations. Collecting this information belongs in the AI implementation plan, with time and people assigned to it.

The queue should not be a surprise discovered after launch. Work through the demand, review effort, and available capacity while you can still change the plan. The people responsible for the decisions deserve that preparation before more work starts arriving.

Try the Review Capacity Lab

The Review Capacity Lab lets you work through these choices in your browser. Start with the article example, then change the number of reviewers or their protected hours. Load the exact-capacity example and notice that its starting queue never clears. Add re-review and watch the workload change.

You can enter your own assumptions, inspect the daily table, and export the scenario and workload timeline. The staffing calculation distinguishes keeping up with new work from clearing the starting queue within the chosen window.

The lab uses averaged daily totals. It does not predict actual waiting times, reviewer fatigue, decision quality, or the effect of a morning burst of difficult cases. Its case-equivalent count expresses remaining workload; it is not a count of partly approved real decisions. A result showing spare hours is a reason to investigate the workflow, not permission to increase an agent’s authority.

Use actual queue records to improve the assumptions. How many requests arrived? How many needed review? How long did the complete review take? How much work returned? How old were the cases when someone finally decided them?

Then compare those observations with the amount of review time the organization protected.

The practical response may be to improve the evidence package so people can review it more efficiently. It may be to narrow the work admitted to the automated path, allocate more qualified review time, or improve independent checks before reconsidering which cases require human attention. Required approvals still apply. Every proposed change needs its own evidence.

There is a human commitment inside this calculation. If a person is responsible for a decision, the organization owes that person a workable opportunity to understand it, challenge it, and refuse it.

Before the next batch starts, decide how much work can reach the review queue. Then protect the time in which someone qualified can finish the decision.

Your AI Passed the Test. Did It Answer the Business Question?

2026-11-01T09:00:00-08:00

Your AI Passed the Test. Did It Answer the Business Question?

Before a product team puts an AI assistant into production, it needs a way to determine whether the assistant can be hired to do the job. That is what evaluations, usually shortened to evals, are for. Evals are different the the tests used in traditional software development, and they are different then the performance metrics used to assess human employees. They are also not user acceptance tests.

An eval is a structured test: give the AI a task, observe what it does, and compare the result with expectations established beforehand. Success might mean answering correctly, completing an action, asking a necessary question, or recognizing that the available information cannot support a conclusion.

Evals are more than a successful demonstration or a score showing how quickly an assistant works. They can include familiar software tests and support user acceptance testing, the testing is not about getting the answer right every time, but to understand its behavior under different circumstances. In other words, they examine how the AI handles a range of tasks and conditions. They provide evidence for deciding where the assistant is dependablly competent to do the job following the right process, where it needs help, and what must improve before release.

Traditional Concepts	What this means in an AI World
Traditional software tests	Evals can include these, but an agent may have several acceptable answers or ways to complete a task. Checking for one exact output may miss whether it accomplished the work.
User acceptance testing	Evals can supply evidence for UAT. They are not synonymous with stakeholder approval, and their usefulness continues after release.
An employee performance review	Business requirements and outcome measures can carry over. But an eval assesses a particular combination of model, instructions, tools and data—not a person’s effort, motivation or general professional competence.
A productivity dashboard	Speed, cost and output volume can be measured. Those measures alone do not establish that the agent completed the right work correctly.
A successful demonstration	A convincing example shows that something worked on that occasion. A useful eval examines a defined collection of cases, including difficult situations and failures, often through repeated attempts.
Training	Running an eval measures performance. It does not, by itself, teach the model or repair the product. The results help people decide what to change.
A certificate of reliability	Passing provides evidence about the cases and criteria tested. It does not guarantee success on unfamiliar tasks or after the system changes.

Teams collect these test cases and run them against different versions of their product. The results help them identify weaknesses that need attention, compare changes to the application or the model over time, and decide what portions of the application are ready for release. The process is repeatable, although the AI’s responses may vary between attempts. Anthropic’s guide to evaluations describes these building blocks.

Consider a fictional company developing an assistant that answers questions about cloud spending. The documents show $120,000 was spent for systems used by customers and employees, $30,000 for software development, and $10,000 for testing. Together, they cover all company cloud costs for the same quarter.

When the assistant is asked to provide a figure for total spending, the assistant should answer $160,000.

Now change the test. Give the assistant only the $120,000 report, which explicitly excludes development and testing. It answers:

“Our total cloud spending last quarter was $120,000.”

That number is present in the report and approves the response.

The number is correct. But is there any support for that conclusion? The company-wide claim is unsupported.

A simple checker is plainly inadequate: it verifies that a value appears in a source without checking whether the source supports the complete claim. But the example exposes a broader design problem. An eval can correctly enforce its scoring rules while those rules fail to capture what the business needs.

A more capable grader helps only if the evaluation gives it the right job.

In Right Numbers, Wrong Questions, posted October 28, 2026, TIFIN researchers examined constructed Indian mutual-fund questions involving ambiguous figures, product variants and missing investor information.

They also tested an AI judge. Providing a dictionary of field definitions helped it identify 40 of 44 mislabeled figures. Yet, without being told the intended interpretation, it accepted 42 of 52 answers that silently chose one reading of a vague question.

This was their key finding. A judge assessing whether an answer addresses “the question as asked” can accept a plausible reading without requiring the assistant to disclose that reading. Better definitions improved one check; they did not establish that the answer matched the user’s unstated intention. Requiring explicit interpretations was proposed, but not tested as a remedy. These were controlled research cases, not failure rates for deployed businesses.

Other research posted that day supplies related evidence. Beyond fabrication found numerical claims in AI-written research that used existing values from the wrong experiment, metric or run. DeoGen found that models performed substantially worse when policy questions lacked enough information for a conclusion. That study used generated cases from four U.S. policy texts, with extended reasoning disabled.

These studies examine different failures. They do not establish how often the problem occurs across enterprises or prove that particular remedies will work in production. They give product teams specific behaviors to investigate: preserving a figure’s meaning, attributing it to the correct result, and recognizing when a necessary fact is missing.

In The Map Is Not the Decision, I argued that analytical outputs need explicit assumptions and boundaries around what they can support. Evaluation makes that requirement concrete. A passing result should depend on whether the answer stays within those boundaries.

For a product team, the work begins before the AI runs.

Someone must assemble representative questions, the information available for each, and the behavior that the job was completed successfully. A test runner can then submit those cases to the product, capture its responses and actions, apply grading rules, and record the results. After a change, the team runs the cases again and compares performance. OpenAI’s evaluation guidance recommends this cycle of defined objectives, representative cases, measurement and continued evaluation.

The grader might be ordinary software checking a calculation, a person assessing an explanation, or another AI applying written criteria. Different parts of the same task can use different graders. An assistant that updates a record should be checked against the resulting record, as well as what it says it did. These distinctions are covered in Anthropic’s evaluation guide.

For the fictional cloud-cost assistant, one test result could look like this:

Part of the test	What it records
User’s question	What was total company cloud spending last quarter?
Available evidence	A report covering only production costs: $120,000.
Required behavior	Identify the limited scope; retrieve missing costs if authorized and available, or explain that the total cannot yet be established.
Assistant’s answer	Total company spending was $120,000.
Grade	Fail: a partial figure was presented as a complete total.
Next action	Investigate the retrieved records, instructions and grading rules before approving the change.

The failure identifies behavior that needs attention. It does not, by itself, establish the cause. Perhaps the search process missed available records. Perhaps the assistant ignored the report’s exclusions. Perhaps the product never specified what to do with incomplete information.

Running an eval does not repair any of those problems. It gives the team evidence with which to investigate them.

This is why domain specialists belong in evaluation design. Engineers can build the machinery, while finance, operations or customer-support specialists help establish what a correct result means. If finance defines spending as payments made and procurement means purchase orders issued, the product team must resolve that difference—or specify when the assistant should present both measures.

As I argued in How to Communicate in a World of AI, specifications preserve intent. Here, that intent must reach the grading rules. “Be accurate” leaves too much unstated. “Do not present production-only costs as a company-wide total” is a requirement someone can test.

The grader also needs scrutiny. Give it a well-cited answer from the wrong quarter. Give it a correct answer phrased differently from the reference answer. Check whether it rejects the misleading response and accepts the valid one.

Then connect those results to a decision.

Suppose the team is considering a faster, cheaper version of the cloud-cost assistant. Its overall evaluation score improves, but it now presents production-only costs as total company spending.

Before testing, the product owner and finance specialist agreed that this failure would block release. The team therefore holds the update, investigates the defect, makes a change, and reruns the relevant tests. It also checks that the assistant still answers directly when complete records are available.

This is a hypothetical release decision, but it illustrates the practical value of evals. An aggregate improvement does not settle whether a change meets the product’s requirements. The team needs to know which behaviors improved, which deteriorated, and whether any failure violates a requirement it considers essential.

Those requirements should reflect the consequences of the work. A confusing sentence and an unsupported financial total need not carry the same weight.

There is also a cost to excessive caution. The financial-query study found that one intervention reduced wrong answers while introducing unnecessary clarification. Stronger models handled some missing-information tasks successfully. More questions were not a universal improvement. Right Numbers, Wrong Questions.

For our cloud-cost assistant, the team should therefore test both conditions: incomplete records should prevent an unsupported total; complete records should produce a direct answer. If an approved definition of spending already exists, the assistant should use it rather than repeatedly asking the user to define the term.

The team can track consequential errors, unnecessary questions, unresolved requests and time to completion separately. That makes it easier to see whether a proposed improvement helps users finish their work or merely transfers more effort to them.

A passing evaluation remains bounded evidence. It shows how the system performed on selected cases under particular grading rules. It does not guarantee every future response. A live check of an individual answer is a separate control, whose effectiveness can itself be evaluated.

These tests also need to continue as the product changes. A different model, revised instruction, new search process or renamed data field can alter behavior. As Dev, Test, and Prod Still Matter: What Gets Deployed Has Changed explains, assurance needs to follow the system being deployed.

Try it: follow an answer from evidence to release

The Founding Documents AI Evaluation Lab puts these ideas into practice with public U.S. documents. Its first exercise confuses amendments proposed in 1789 with those ratified in 1791. Follow the retrieved passage, compare the graders, and decide whether the answer meets the requirement for release.

Interactive lab: follow the evidence and test the release decision Six short lessons · about 20 minutes · no API key needed for replay

Start in Replay with Candidate selected. This version passes more cases overall but fails one requirement that blocks release.

Open the action trace and follow the citation to the passage the agent retrieved.
Compare the weak checker with the task-specific grades. Notice which question each one answers.
Choose Decide what can ship, then switch the agent version to Revised. Compare the results before changing the release policy.

Replay uses authored teaching responses scored with Microsoft Agent Framework. Optional live runs use your own model connection. The lab also includes Python and React downloads for your own experiments.

On a small screen, use the link below to give the lab's evidence and controls more room.

Open the lab in a new tab

Select Interactive lab above again to collapse this exercise and return to the article.

A useful starting point is one answer the product currently passes. Change the scope of its evidence without changing the number. Remove a fact required for its conclusion. Then restore complete information.

Check whether the product limits its claims when the evidence is incomplete—and answers directly when the evidence is sufficient.

REPRINT: Two views of programming

2026-10-25T00:00:00-07:00

Source transcription

Archive transcription by Jorge A. Pérez Parra; revised March 25, 2005.

by Edsger W. Dijkstra

In the world around us we encounter two radically different views of programming:

View A: Programming in essence is very easy.
View B: Programming is intrinsically very difficult.

One can dismiss this discrepancy by concluding that, apparently, in the two views the same word “programming” is used in two quite different meanings, and then return to the order of the day. Whether view A or view B is the predominant one, however, has a deep influence, not only on the personnel policy of computer using organizations and on the curriculum policy of our educational institutes, but even on the direction of development and research in computing science itself. It seems, therefore, worthwhile to explore the nature of the difference between the two meanings and to identify, if possible, the underlying assumptions that would make each of them appropriate. To do so is the purpose of this paper.

In this exploration I have what could be regarded as a handicap: in the controversy I am not neutral at all. I am a strong supporter of view B and regard view A as the underlying cause of many mistakes. On the other hand I don’t think that having an opinion disqualifies me as an author, certainly if I warn my readers in advance and do not feign a fake neutrality. As our analysis proceeds we shall discover how these different views of programming (which is a human activity!) are related to different types of Man. This, all by itself, is already a valuable insight, as it explains the nearly religious fervour with which the battle between the defenders of the opposing views —creeds?— is sometimes fought.

The early history of automatic computing makes view A only too understandable. Before we had computers, programming was no problem at all. Then came the first machines: compared with the machines we have now they were mere toys, and, compared with what we try to do now, they were used for “micro-applications” only. If at that stage programming was a problem, it was only a mild one. Add to this the sources of difficulties that at that time absorbed —or should we say in retrospect: usurped?— the major part of our attention:

arithmetic units were slow with respect to what we wanted to do: that shoe pinched nearly always, and in the name of program efficiency all possible coding tricks were permitted (and very few were not applied)
design and construction of arithmetic units were such a novel and, therefore, difficult task that if a next anomaly in the instruction code could save a number of flip-flops, the flip-flops were usually saved —also, of course, because we had so little programming experience that we could not recognize “anomalies in the instruction code” too well—; as a result there was, besides pressure to apply coding tricks, also a great opportunity for doing so
stores were always too small, a pinching shoe that, together with the general unreliability of the first hardware, prohibited more sophisticated ways of machine usage.

In that time programming presented itself primarily as battle against the machine’s limitations, a battle that was to be won by a cunning, be it not very systematic, exploitation of each machine’s specific properties: it was the heyday of the virtuoso coder.

In the next to ten to fifteen years processing units became a thousand times faster, stores became a thousand times larger, and high-level programming languages came into general use. And it was during that period, when on the one hand programming was still firmly associated with the pinching shoe, while on the other hand the shoe was felt to pinch less and less, that it was expected that with another five years of technical progress the problems of programming would have disappeared. It was during that period that view A was born. It was at the end of that period that, inspired by view A, COBOL was designed with the avowed intention that it should make programming by professional programmers superfluous by allowing the “user” —was it at that time that the term “user” gained currency?— to write down what he wanted in “plain English” that every one could read and understand.

We all know, that that splendid dream did not come true. The next five years gave us, instead of the disappearance of all programming problems, the software crisis, and COBOL, instead of doing away with professional programmers, became the major programming vehicle for ever growing numbers of them; and another ten years later we still have machines with which flaws in the basic software cause on the average one hour down-time for every fifteen hours of production. There are, apparently, still serious programming problems ….

The amazing thing is that, in spite of the overwhelming evidence to the contrary, view A survived. In explaining that amazing thing fact some point with a somewhat accusing finger to large organizations, either to computer using ones that, having atttacted a large labour force based in view A, thereby have lost the freedom to part from it, or to computer manufacturers and educational institutes that promote a widely-held view A, that they are supposed to suppose essential for their market. Even if that finger is not raised without some justification, I just cannot accept it as a full explanation of view A’s survival, and must assume that view A satisfies some deeper, psychological needs.

How emerged view B? There were people who felt that the advent of faster and bigger machines would replace the pinching shoe at most by a fitting shoe, and that, therefore, the economics of program execution would remain a serious concern of the programmer, a concern that would even become more important as size of machines and applications would grow, and, with more complex installations, would pose more difficult problems. Also it was observed that switching from machine code to a high-level programming language did not guarantee all the benefits that were hoped for. In particular, programmers still produced, as willingly as before, large chunks of ununderstandable code, the only difference being that now they did it on a more grandiose scale, and that high-level bugs had replaced low-level ones. They also realized that the advent of high-level programming languages had not reduced the essential need for accuracy: redundancy in high-level programming languages only reduces the ill effects of some inaccuracies. And thus view B was born. (View B is NOT the reaction to the software crisis that surfaced in 1968, for it is many years older. View B had, in fact, predicted that crisis, but even that confirmation has not killed view A.)

After this interlude about the emerging of view B, we return to our question, how and why, face to face with the undeniable software problems, view A, viz. that programming is in essence very easy, survives. The answers is: by faith, not faith in better programmers, but faith in better programming languages or (conversational?) programming services, and faith in better management techniques.

I happen to be of the considered opinion that programming is one of the more difficult branches of applied mathematics, because it is also one of the more difficult branches of engineering, and vice versa. When I tried to explain to one of my mathematical colleagues why I held that view, he bluntly refused to listen to my arguments and, instead, blamed me and my fellow computing scientists for not yet having designed the programming language that would make programming as easy as it should be! Should I have asked him, why mathematicians have not developed a notation that would enable everyone, no matter how ill-equipped otherwise, to do mathematics?

After more probing one discovers that the proponents of view A do not deny the potential complexity of programs, nor the problem of composing them, but believe that life will become easy for the programmer because all the more difficult parts of the task will eventually taken over by the machine. They point to the advent of high-level programming languages that made programming already so much easier than in the old machine code days, and recklessly extrapolate, that in the future programming will become trivial. But is this extrapolation justified? I have programmed extensively, both in machine codes and high-level programming languages, and the latter are undoubtedly more convenient because all sorts of otherwise irrelevant decisions with many clerical consequences —such as details of storage layout— need not be taken explicitly because one accepts the outcome of the compiler’s storage allocation algorithm. The transition to a high level freed us from a number of trivia. In doing so, it has made programming an activity with a smaller component of drudgery, and therefore with a larger component of invention: precisely those parts of the job with one used to fill the day when feeling less bright, are the ones that have disappeared! The conclusion that the advent of high-level programming languages has created the need for programmers of higher intellectual caliber has been fully confirmed by my observations in Western Europe (where I could follow the development at close quarters) where in the late sixties many large computer using organizations had problems in finding appropriate employment for the programmers they had attracted in the fifties, because their profession had outgrown their intellectual capacities.

But neither this observation, nor pointing out COBOL’s failure to do away with professional programmers, makes any impression upon the faithful. They will explain that the traditional high-level programming languages were failures because they were still “procedural”, and that COBOL’s failure is evident because, for lack of interaction, it is not really plain English, but that within five or five or ten years further progress in Artificial Intelligence (AI for the intimi) will enable us to build “context dependent”, “knowledge-based”,”automated systems for reasoning and understanding” such that the “user needs only talk to them”.

I may be an incurable sceptic, but I find it very hard to believe such claims to be justified. They are specific instances of the expectation that we shall witness — I quote from a letter I recently received— “, in general, the assumption by the computer of progressively higher levels of what now is considered human skill, knowledge and intelligence.” It is not my intention of to repeat parts of heated discussions we have had about the significance of Artificial Intelligence (see, for instance, ¹), nor is there any need to do so.

Firstly, looking backwards, one conclusion is certain: confusing AI’s hopes for the future with tomorrow’s reality would be folly, and it would be an act of utter irresponsibility not to prepare ourselves for the case that the dreams of AI will remain dreams as long as we live. Or, to put it in other words, in view of the severity of today’s programming problems, general cautiousness forces us not to discard view B.

Secondly, if his creations are going to be relied upon, it will be the programmer’s primary task to design his artifacts so understandable, that he can take the responsibility for them, and, regardless of the answer to the question how much of his current activity may ultimately be delegated to machines, we should always remember that neither “understanding” nor “being responsible” can properly be classified as activities: they are more like “states of mind” and are intrinsically incapable of being delegated.

I think it unwise, particularly for a computing scientist, to underestimate the influence of that school of psychologist that, because they found the human mind too difficult and elusive an object for their study, turned to the study of rats instead, and even restrict that study —as I saw expressed recently— “to the most mechanical forms of behavior —often so mechanical that even rats have no chance to show their higher faculties—”. By presenting their crude, mechanical models as a valid approximation for the human mind, they have blurred the distinction between man and machine dangerously, and we observe the two complementary phenomena: an anthropomorphic view of machines and a mechanical view of people.

This confusion is by no means confined to the high priests of AI. The preponderance of anthropomorphic terminology in computing science in general —”memory”, “interpreter”,”programming language”, “handshaking”, “conversation”, to mention but a few— is a warning not to be ignored. I would not know how to think and talk without metaphors; I also know that each metaphor carries with it the danger of false connotations. In the case of the anthropomorphic terminology in computer science we have since long reached the stage that the dangers of confusion far outweigh the benefits of the analogy.

Also the mechanical view of people seems among computing scientist (and their managers) more widely spread than I consider healthy. For I suspect that it is this mechanical view that restricts the activity of programmers to the mechanical activity of writing code, and then measures “programmer productivity” by the number of lines of code produced. (When a very well-known and widely respected computer scientist recently used that measure for programmer productivity in a lecture, the suggestion came from the audience that, instead of talking about “the lines of code produced” we should talk about “the lines of code used” and that, therefore, the speaker was booking them on the wrong side of the ledger. The speaker answered that he stuck to his productivity measure, because he did not know of any alternatives that allowed proper quantification!) This can no longer be classified as a harmless mistake, for the adoption of that nonsensical “productivity measure” for grading programmers is guaranteed to promote the writing of insipid code.

The influence of psychology has been brought into the picture because it explains the tenacity with which so many people cling to view A.

It is not so much the computer manufacturers, that want to do as if they sell an easy product; it is not so much the managers of software projects, that would like to view the programming activity as a simple and predictable one; it is not so much our educational institutes, that would like to train their students with predictable success.

It is the comfortable illusion of Man as elaborate automata that, like a drug, seems to have freed its victims from the burden of responsibility. Accepting programming as a hard intellectual challenge would place the full weight of that burden back upon their shoulders.

Plataanstraat 5
NL–4565 NUENEN
The Netherlands

prof.dr.Edsger W.Dijkstra
Burroughs Research Fellow

Flowers, B.H. “Artificial Intelligence: a paper symposium” April 1973, Science Research Council, State House, High Holborn, London WC1R 4TA. ↩

REPRINT: No Silver Bullet

2026-10-25T00:00:00-07:00

Source PDF

There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity.

Frederick P. Brooks, Jr. - University of North Carolina at Chapel Hill ¹

Abstract

All software construction involves essential tasks, the fashioning of the complex conceptual structures that compose the abstract software entity, and accidental tasks, the representation of these abstract entities in programming languages and the mapping of these onto machine languages within space and speed constraints. Most of the big past gains in software productivity have come from removing artificial barriers that have made the accidental tasks inordinately hard, such as severe hardware constraints, awkward programming languages, lack of machine time. How much of what software engineers now do is still devoted to the accidental, as opposed to the essential? Unless it is more than 9/10 of all effort, shrinking all the accidental activities to zero time will not give an order of magnitude improvement.

Therefore it appears that the time has come to address the essential parts of the software task, those concerned with fashioning abstract conceptual structures of great complexity. I suggest:

Exploiting the mass market to avoid constructing what can be bought.
Using rapid prototyping as part of a planned iteration in establishing software requirements.
Growing software organically, adding more and more function to systems as they are run, used, and tested.
Identifying and developing the great conceptual designers of the rising generation.

Introduction

Of all the monsters who fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, we seek bullets of silver than can magically lay them to rest.

The familiar software project has something of this character (at least as seen by the non-technical manager), usually innocent and straightforward, but capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet, something to make software costs drop as rapidly as computer hardware costs do.

But, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or management technique, which by itself promises even one order of magnitude improvement in productivity, in reliability, in simplicity. In this chapter we shall try to see why, but examining both the nature of the software problem and the properties of the bullets proposed.

Skepticism is not pessimism, however. Although we see no startling breakthroughs, and indeed, believe such to be inconsistent with the nature of software, many encouraging innovations are under way. A disciplined, consistent effort to develop, propagate, and exploit them should indeed yield an order-of-magnitude improvement. There is no royal road, but there is a road.

The first step toward the management of disease was replacement of demon theories and humours theories by the germ theory. That very step, the beginning of hope, in itself dashed all hopes of magical solutions. It told workers the progress would be made stepwise, at great effort, and that a persistent, unremitting care would have to be paid to a discipline of cleanliness. So it is with software engineering today.

Does It Have To Be Hard? – Essential Difficulties

Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any−no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware. We cannot expect ever to see twofold gains every two years.

First, we must observe that the anomaly is not that software progress is so slow but that computer hardware progress is so fast. No other technology since civilization began has seen six orders of magnitude price-performance gain in 30 years. In no other technology can one choose to take the gain in either improved performance or in reduced costs. These gains flow from the transformation of computer manufacture from an assembly industry into a process industry.

Second, to see what rate of progress we can expect in software technology, let us examine its difficulties. Following Aristotle, I divide them into essence−the difficulties inherent in the nature of the software−and accidents−those difficulties that today attend its production but that are not inherent.

The accidents I discuss in the next section. First let us consider the essence.

The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract, in that the conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed.

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared to the conceptual errors in most systems.

If this is true, building software will always be hard. There is inherently no silver bullet.

Let us consider the inherent properties of this irreducible essence of modern software systems: complexity, conformity, changeability, and invisibility.

Complexity. Software entities are more complex for their size than perhaps any other human construct, because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into one, a subroutine, open or closed. In this respect software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound.

Digital computers are themselves more complex than most things people build; they have very large numbers of states. This makes conceiving, describing, and testing them hard. Software systems have orders of magnitude more states than computers do.

Likewise, a scaling-up of a software entity is not merely a repetition of the same elements in larger size; it is necessarily an increase in the number of different elements. In most cases, the elements interact with each other in some nonlinear fashion, and the complexity of the whole increases much more than linearly.

The complexity of software is in essential property, not an accidental one. Hence descriptions of a software entity that abstract away its complexity often abstract away its essence. Mathematics and the physical sciences made great strides for three centuries by constructing simplified models of complex phenomena, deriving properties from the models, and verifying those properties experimentally. This worked because the complexities ignored in the models were not the essential properties of the phenomena. It does not work when the complexities are the essence.

Many of the classical problems of developing software products derived from this essential complexity and its nonlinear increased with size. From the complexity comes the difficulty of communication among team members, which leads to product flaws, cost overruns, schedule delays. From the complexity comes the difficulty of enumerating, much less understanding, all the possible states of the program, and from that comes the unreliability. From the complexity of the functions comes the difficulty of invoking those functions, which makes programs hard to use. From complexity of structure comes the difficulty of extending programs to new functions without creating side effects. From the complexity of structure comes the unvisualized state that that constitute security trapdoors.

Not only technical problems but management problems as well come from the complexity. This complexity makes overview hard, thus impeding conceptual integrity. It makes it hard to find and control all the loose ends. It creates the tremendous learning and understanding burden that makes personnel turnover a disaster.

Conformity. Software people are not alone in facing complexity. Physics deals with terribly complex objects even at the “fundamental” particle level. The physicist labors on, however, in a firm faith that there are unifying principles to be found, whether in quarks or in unified field theories. Einstein repeatedly argued that there must be simplified explanations of nature, because God is not capricious or arbitrary.

No such faith comforts the software engineer. Much of the complexity he must master is arbitrary complexity, forced without rhyme or reason by the many human institutions and systems to which his interfaces must confirm. These differ from interface to interface, and from time to time, not because of necessity but only because they were designed by different people, rather than by God.

In many cases the software must confirm because it has most recently come to the scene. In others it must conform because it is perceived as the most conformable. But in all cases, much complexity comes from conformation to other interfaces; this cannot be simplified out by any redesign of the software alone.

Changeability. The software entity is constantly subject to pressures for change. Of course, so are buildings, cars, and computers. But manufactured things are infrequently changed after manufacture; they are superseded by later models, or essential changes are incorporated in later serial-number copies of the same basic design. Callbacks of automobiles are really quite infrequent; field changes of computers somewhat less so. Both are much less frequent than modifications to fielded software.

Partly this is because the software in a system embodies its function, and the function is the part that most feels the pressures of change. Partly it is because software can be changed more easily−it is pure thought-stuff, infinitely malleable. Buildings do in fact get changed, but the high costs of change, understood by all, serve to dampen the whim of the changers.

All successful software gets changed. Two processes are at work. As a software product is found to be useful, people try it in new cases at the edge of, or beyond, the original domain. The pressures for extended function come chiefly from users who like the basic function and invent new uses for it.

Second, successful software also survives beyond the normal life of the machine vehicle for which it is first written. If not new computers, then at least new disks, new displays, new printers come along; and the software must be conformed to its new vehicles of opportunity.

In short, the software product is embedded in a cultural matrix of applications, users, laws, and machine vehicles. These all change continually, and their changes inexorably force change upon the software product.

Invisibility. Software is invisible and unvisualizable. Geometric abstractions are powerful tools. The floor plan of a building helps both architect and client evaluate spaces, traffic flows, and views. Contradictions become obvious, omissions can be caught. Scale drawings of mechanical parts and stick-figure models of molecules, although abstractions, serve the same purpose. A geometric reality is captured in a geometric abstraction.

The reality of software is not inherently embedded in space. Hence it has no ready geometric representation in the way that land has maps, silicon chips have diagrams, computers have connectivity schematics. As soon as we attempt to diagram software structure, we find it to constitute not one, but several, general directed graphs, superimposed one upon another. The several graphs may represent the flow of control, the flow of data, patterns of dependency, time sequence, name-space relationships. These are usually not even planar, much less hierarchical. Indeed, one of the ways of establishing conceptual control over such structure is to enforce link cutting until one or more of the graphs becomes hierarchical.²

In spite of progress in restricting and simplifying the structures of software, they remain inherently unvisualizable, thus depriving the mind of some of its most powerful conceptual tools. This lack not only impedes the process of design within one mind, it severely hinders communication among minds.

Past Breakthroughs Solved Accidental Difficulties

If we examine the three steps in software technology that have been most fruitful in the past, we discover that each attacked a different major difficulty in building software, but they have been the accidental, not the essential, difficulties. We can also see the natural limits to the extrapolation of each such attack.

High-level languages. Surely the most powerful stroke for software productivity, reliability, and simplicity has been the progressive use of high-level languages for programming. Most observers credit that development with at least a factor of five in productivity, and with concomitant gains in reliability, simplicity, and comprehensibility.

What does a high-level language accomplish? It frees a program from much of its accidental complexity. An abstract program consists of conceptual constructs: operations, data types, sequences, and communication. The concrete machine program is concerned with bits, registers, conditions, branches, channels, disks, and such. To the extent that the high-level language embodies the constructs wanted in the abstract program and avoids all lower ones, it eliminates a whole level of complexity that was never inherent in the program at all.

The most a high-level language can do is to furnish all the constructs the programmer imagines in the abstract program. To be sure, the level of our sophistication in thinking about data structures, data types, and operations is steadily rising, but at an ever-decreasing rate. And language development approaches closer and closer to the sophistication of users.

Moreover, at some point the elaboration of a high-level language becomes a burden that increases, not reduces, the intellectual task of the user who rarely uses the esoteric constructs.

Time-sharing. Most observers credit time-sharing with a major improvement in the productivity of programmers and in the quality of their product, although not so large as that brought by high-level languages.

Time-sharing attacks a distinctly different difficulty. Time-sharing preserves immediacy, and hence enables us to maintain an overview of complexity. The slow turnaround of batch programming means that we inevitably forget the minutiae, if not the very thrust, of what we were thinking when we stopped programming and called for compilation and execution. This interruption of consciousness is costly in time, for we must refresh. The most serious effect may well be the decay of grasp of all that is going on in a complex system.

Slow turn-around, like machine-language complexities, is an accidental rather than an essential difficulty of the software process. The limits of the contribution of time-sharing derive directly. The principle effect is to shorten system response time. As it goes to zero, at some point it passes the human threshold of noticeability, about 100 milliseconds. Beyond that no benefits are to be expected.

Unified programming environments. Unix and Interlisp, the first integrated programming environments to come into widespread use, are perceived to have improved productivity by integral factors. Why?

They attack the accidental difficulties of using programs together, by providing integrated libraries, unified file formats, and piles and filters. As a result, conceptual structures that in principle could always call, feed, and use one another can indeed easily do so in practice.

This breakthrough in turn stimulated the development of whole toolbenches, since each new tool could be applied to any programs by using the standard formats.

Because of these successes, environments are the subject of much of today’s software engineering research. We will look at their promise and limitations in the next section.

Hopes for the Silver

Now lets us consider the technical developments that are most often advanced as potential silver bullets. What problems do they address? Are they the problems of essence, or are they remainders of our accidental difficulties? Do they offer revolutionary advances, or incremental ones?

Ada and other high-level language advances. One of the most touted recent developments is the programming language Ada, a general-purpose, high-level language of the 1980s. Ada indeed not only reflects evolutionary improvements in language concepts but embodies features to encourage modern design and modularization concepts. Perhaps the Ada philosophy is more of an advance than the Ada language, for it is the philosophy of modularization, of abstract data types, of hierarchical structuring. Ada is perhaps over-rich, the natural product of the process by which requirements were laid on its design. That is not fatal, for subset working vocabularies can solve the learning problem, and hardware advances will give us the cheap MIPS to pay for the compiling costs. Advancing the structuring of software systems is indeed a very good use for the increased MIPS our dollars will buy. Operating systems, loudly decried in the 1960s for their memory and cycle costs, have proved to be an excellent form in which to use some of the MIPS and cheap memory bytes of the past hardware surge.

Nevertheless, Ada will not prove to be the silver bullet that slays the software productivity monster. It is, after all, just another high-level language, and the biggest payoff from such languages came from the first transition, up from the accidental complexities of the machine into the more abstract statement of step-by-step solutions. Once those accidents have been removed, the remaining ones are smaller, and the payoff from their removal will surely be less.

I predict that a decade from now, when the effectiveness of Ada is assessed, it will be seen to have made a substantial difference, but not because of any particular language feature, nor indeed because of all of them combined. Neither will the new Ada environment prove to be the cause of the improvements. Ada’s greatest contribution will be that switching to it occasioned training programmers in modern software design techniques.

Object-oriented programming. Many students of the art hold out more hope for object-oriented programming than for any of the other technical fads of the day.³ I am among them. Mark Sherman of Dartmouth notes that we must be careful to distinguish two separate ideas that go under that name: abstract data types and hierarchical types, also called classes. The concept of the abstract data type is that an object’s type should be defined by a name, a set of proper values, and a set of proper operations, rather than its storage structure, which should be hidden. Examples are Ada packages (with private types) or Modula’s modules.

Hierarchical types, such as Simula-67’s classes, allow the definition of general interfaces that can be further refined by providing subordinate types. The two concepts are orthogonal−there may be hierarchies without hiding and hiding without hierarchies. Both concepts represent real advances in the art of building software.

Each removes one more accidental difficulty from the process, allowing the designer to express the essence of his design without having to express large amounts of syntactic material that add no new information content. For both abstract types and hierarchical types, the result is to remove a higher-order sort of accidental difficulty and allow a higher-order expression of design.

Nevertheless, such advances can do no more than to remove all the accidental difficulties from the expression of the design. The complexity of the design itself is essential; and such attacks make no change whatever in that. An order-of-magnitude gain can be made by object-oriented programming only if the unnecessary underbrush of type specification remaining today in our programming language is itself responsible for nine-tenths of the work involved in designing a program product. I doubt it.

Artificial intelligence. Many people expect advances in artificial intelligence to provide the revolutionary breakthrough that will give order-of-magnitude gains in software productivity and quality.⁴ I do not. To see why, we must dissect what is meant by “artificial intelligence” and then see how it applies.

Parnas has clarified the terminological chaos:

Two quite different definitions of AI are in common use today. AI-1: The use of computers to solve problems that previously could only be solved by applying human intelligence. AI-2: The use of a specific set of programming techniques knows as heuristic or rule-based programming. In this approach human experts are studies to determine what heuristics or rules of thumb they use in solving problems. . . . The program is designed to solve a problem the way that humans seem to solve it.

The first definition has a sliding meaning. . . . Something can fit the definition of AI-1 today but, once we see how the program works and understand the problem, we will not think of it as AI anymore. . . . Unfortunately I cannot identify a body of technology that is unique to this field. . . . Most of the work is problem-specific, and some abstraction or creativity is require to see how to transfer it.⁵

I agree completely with this critique. The techniques used for speech recognition seem to have little in common with those used for image recognition, and both are different from those used in expert systems. I have a hard time seeing how image recognition, for example, will make any appreciable difference in programming practice. The same is try of speech recognition. The hard thing about building software is deciding what to say, not saying it. No facilitation of expression can give more than marginal gains.

Expert systems technology, AI-2, deserves a section of its own.

Expert systems. The most advanced part of the artificial intelligence art, and the most widely applies, is the technology for building expert systems. Many software scientists are hard at work applying this technology to the software-building environment.⁶ What is the concept, and what are the prospects?

An expert system is a program containing a generalized inference engine and a rule base, designed to take input data and assumptions and explore the logical consequences through the inferences derivable from the rule base, yielding conclusions and advice, and offering to explain its results by retracing its reasoning for the user. The inference engines typically can deal with fuzzy or probabilistic data and rules in addition to purely deterministic logic.

Such systems offer some clear advantages over programmed algorithms for arriving at the same solutions to the same problems:

Inference engine technology is developed in an application-independent way, and then applied to many uses. One can justify much more effort on the inference engines. Indeed, that technology is well advanced.
The changeable parts of the application-peculiar materials are encoded in the rule base in a uniform fashion, and tools are provided for developing, changing, testing, and documenting the rule base. This regularizes much of the complexity of the application itself.

Edward Feigenbaum says that the power of such systems does not come from ever-fancier inference mechanisms, but rather from ever-richer knowledge bases that reflect the real world more accurately. I believe the most important advance offered by the technology is the separation of the application complexity from the program itself.

How can this be applied to the software task? In many ways: suggesting interface rules, advising on testing strategies, remembering but-type frequencies, offering optimization hints, etc.

Consider an imaginary testing advisor, for example. In its most rudimentary form, the diagnostic expert system is very like a pilot’s checklist, fundamentally offering suggestions as to possible causes of difficulty. As the rule base is developed, the suggestions become more specific, taking more sophisticated account of the trouble symptoms reported. One can visualize a debugging assistant that offers very generalized suggestions at first, but as more and more system structure is embodied in the rule base, comes more and more particular in the hypotheses is generates and the tests it recommends. Such an expert system may depart most radically from the conventional ones in that its rule base should probably be hierarchically modularized in the same way the corresponding software product is, so that as the product is modularly modified, the diagnostic rule base can be modularly modified as well.

The work required to generate the diagnostic rules is work that will have to be done anyway in generating the set of test cases for the modules and for the system. If it is done in a suitably general manner, with a uniform structure for rules and a good inference engine available, it may actually reduce the total labor of generating bring-up test cases, as well as helping in lifelong maintenance and modification testing. In the same way, we can postulate other advisors probably many of them and probably simple ones for the other parts of the software construction task.

Many difficulties stand in the way of early realization of useful expert advisors to the program developer. A crucial part of our imaginary scenario is the development of easy ways to get from program structure specification to the automatic or semi-automatic generation of diagnostic rules. Even more difficult and important is the twofold task of knowledge acquisition: finding articulate, self-analytical experts who know why they do things; and developing efficient techniques for extracting what they know and distilling it into rule bases. The essential prerequisite for building an expert system is to have an expert.

The most powerful contribution of expert systems will surely be to put at the service of the inexperienced programmer the experience and accumulated wisdom of the best programmers. This is no small contribution. The gap between the best software engineering practice and the average practice is very wide−perhaps wider than in any other engineering discipline. A tool that disseminates good practice would be important.

“Automatic” programming. For almost 40 years, people have been anticipating and writing about “automatic programming”, the generation of a program for solving a problem from a statement of the problem specifications. Some people today write as if they expected this technology to provide the next breakthrough.⁷

Parnas implies that the term is used for glamour and not semantic content, asserting,

In short, automatic programming always has been a euphemism for programming with a higher-level language than was presently available to the programmer.⁸

He argues, in essence, that in most cases it is the solution method, not the problem, whose specification has to be given.

Exceptions can be found. The technique of building generators is very powerful, and it is routinely used to good advantage in programs for sorting. Some systems for integrating differential equations have also permitted direct specification of the problem. The system assessed the parameters, chose from a library of methods of solution, and generated the programs.

These applications have very favorable properties:
The problems are readily characterized by relatively few parameters.
There are many known methods of solution to provide a library of alternatives.
Extensive analysis has led to explicit rules for selecting solution techniques, given problem parameters.

It is hard to see how such techniques generalize to the wider world of the ordinary software system, where cases with such neat properties are the exception. It is hard even to imagine how this breakthrough in generalization could conceivably occur.

Graphical programming. A favorite subject for PH.D. dissertations in software engineering is graphical, or visual, programming, the application of computer graphics to software design.⁹ Sometimes the promise of such an approach is postulated from the analogy with VLSI chip design, where computer graphics plays so fruitful a role. Sometimes the approach is justified by considering flowcharts as the ideal program design medium, and providing powerful facilities for constructing them.

Nothing even convincing, much less exciting, has yet emerged from such efforts. I am persuaded that nothing will.

In the first place, as I have argued elsewhere, the flow chart is a very poor abstraction of software structure.¹⁰ Indeed, it is best viewed as Burks, von Neumann, and Goldstine’s attempt to provide a desperately needed high-level control language for their proposed computer. In the pitiful, multipage, connection-boxed form to which the flow chart has today been elaborated, it has proved to be essentially useless as a design-tool programmers draw flow charts after, not before, writing the programs they describe.

Second, the screens of today are too small, in pixels, to show both the scope and the resolution of any serious detailed software diagram. The so-called “desktop metaphor” of today’s workstation is instead an “airplane-seat” metaphor. Anyone who has shuffled a lapful of papers while seated in a coach between two portly passengers will recognize the difference−one can see only a very few things at once. The true desktop provides overview of and random access to a score of pages. Moreover, when fits of creativity run strong, more than one programmer or writer has been known to abandon the desktop for the more spacious floor. The hardware technology will have to advance quite substantially before the scope of our scopes is sufficient to the software design task.

More fundamentally, as I have argued above, software is very difficult to visualize. Whether we diagram control flow, variable scope nesting, variable cross-references, data blow, hierarchical data structures, or whatever, we feel only one dimension of the intricately interlocked software elephant. If we superimpose all the diagrams generated by the many relevant views, it is difficult to extract any global overview. The VLSI analogy is fundamentally misleading−a chip design is a layered two-dimensional object whose geometry reflects its essence. A software system is not.

Program verification. Much of the effort in modern programming goes into the testing and repair of bugs. Is there perhaps a silver bullet to be found by eliminating the errors at the source, in the system design phase? Can both productivity and product reliability be radically enhance by following the profoundly different strategy of proving designs correct before the immense effort is poured into implementing and testing them?

I do not believe we will find the magic here. Program verification is a very powerful concept, and it will be very important for such things as secure operating system kernels. The technology does not promise, however, to save labor. Verifications are so much work that only a few substantial programs have ever been verified.

Program verification does not mean error-proof programs. There is no magic here, either. Mathematical proofs also can be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it.

More seriously, even perfect program verification can only establish that a program meets its specification. The hardest part of the software task is arriving at a complete and consistent specification, and much of the essence of building a program is in fact the debugging of the specification.

Environments and tools. How much more gain can be expected from the exploding researches into better programming environments? One’s instinctive reaction is that the big-payoff problems were the first attacked, and have been solved: hierarchical file systems, uniform file formats so as to have uniform program interfaces, and generalized tools. Language-specific smart editors are developments not yet widely used in practice, but the most they promise is freedom from syntactic errors and simple semantic errors.

Perhaps the biggest gain yet to be realized in the programming environment is the use of integrated database systems to keep track of the myriads of details that must be recalled accurately by the individual programmer and kept current in a group of collaborators on a single system.

Surely this work is worthwhile, and surely it will bear some fruit in both productivity and reliability. But by its very nature, the return from now on must be marginal.

Workstations. What gains are to be expected for the software art from the certain and rapid increase in the power and memory capacity of the individual workstation? Well, how many MIPS can one use fruitfully? The composition and editing of programs and documents is fully supported by today’s speeds. Compiling could stand a boost, but a factor of 10 in machine speed would surely leave think-time the dominant activity in the programmer’s day. Indeed, it appears to be so now.

More powerful workstations we surely welcome. Magical enhancements from them we cannot expect.

Promising Attacks on the Conceptual Essence

Even though no technological breakthrough promises to give the soft of magical results with which we are so familiar in the hardware area, there is both an abundance of good working going on now, and the promise of steady, if unspectacular progress.

All of the technological attacks on the accidents of the software process are fundamentally limited by the productivity equation:

Time of task = ∑ (Frequency)ᵢ x (Time)ᵢ

If, as I believe, the conceptual components of the task are now taking most of the time, then no amount of activity on the task components that are merely the expression of the concepts can give large productivity gains.

Hence we must consider those attacks that address the essence of the software problem, the formulation of these complex conceptual structures. Fortunately, some of these are very promising.

Buy versus build. The most radical possible solution for constructing software is not to construct it at all.

Every day this becomes easier, as more and more vendors offer more and better software products for a dizzying variety of applications. While we software engineers have labored on production methodology, the personal computer revolution has created not one, but m any, mass markets for software. Every newsstand carried monthly magazines which, sorted by machine type, advertise and review dozens of products at prices from a few dollars to a few hundred dollars. More specialized sources offer very powerful products for the workstation and other Unix markets. Even software tolls and environments can be bought off-the-shelf. I have elsewhere proposed a market place for individual modules.

Any such product is cheaper to buy than to build afresh. Even at a cost of $100,000, a purchased piece of software is costing only about as much as one programmer-year. And delivery is immediate! Immediate at least for products that really exist, products whose developer can refer the prospect to a happy user. Moreover, such products tend to be much better documented and somewhat better maintained than homegrown software.

The development of the mass market is, I believe, the most profound long-run trend in software engineering. The cost of software has always been development cost, not replication cost. Sharing that cost among even a few users radically cuts the per-user cost. Another way of looking at it is that the use of n copies of a software system effectively multiplies the productivity of its developers by n. That is an enhancement of the productivity of the discipline and of the nation.

The key issue, of course, is applicability. Can I use an available off-the-shelf package to do my task? A surprising thing has happened here. During the 1950s and 1960s, study after study showed that users would not use off-the-shelf packages for payroll, inventory control, accounts receivable, etc. The requirements were too specialized, the case-to-case variation too high. During the 1980s, we find such packages in high demand and widespread use. What has changed?

Not really the packages. They may be somewhat more generalized and somewhat more customizable than formerly, but not much. Not really the applications, either. If anything, the business and scientific needs of today are more diverse, more complicated than those of 20 years ago.

The big change has been in the hardware/software cost ratio. The buyer of a $2-million machine in 1960 felt that he could afford $250,000 more for a customized payroll program, one that slipped easily and nondisruptively into the computer-hostile social environment. Buyers of $50,000 office machines today cannot conceivably afford customized payroll programs; so they adapt their payroll procedures to the packages available. Computers are now so commonplace, if not yet so beloved, that the adaptations are accepted as a matter of course.

There are dramatic exceptions to my argument that the generalization of the software packages has changed little over the years: electronic spreadsheets and simple database systems. These powerful tools, so obvious in retrospect and yet so late appearing, lend themselves to myriad uses, some quite unorthodox. Articles and even books now abound on how to tackle unexpected tasks with the spreadsheet. Large numbers of applications that would formerly have been written as custom programs in Cobol or Report Program Generator are now routinely done with these tools.

Many users now operate their own computers day in and day out on varied applications without ever writing a program. Indeed, many of these users cannot write new programs for their machines, but they are nevertheless adept at solving new problems with them.

I believe the single most powerful software productivity strategy for man organizations to day is to equip the computer-naïve intellectual workers on the firing line with personal computers and good generalized writing, drawing, file and spreadsheet programs, and turn them loose. The same strategy, with simple programming capabilities, will also work for hundreds of laboratory scientists.

Requirements refinement and rapid prototyping. The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is to difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult go rectify later.

Therefore the most important function that software builders do for their clients is the iterative extraction and refinement of the product requirements. For the truth is, the clients do not know what they want. They usually do not know what questions must be answered, and they almost never have thought of the problem in the detail that must be specified. Even the simple answer−”Make the new software system work like our old manual information-processing system” −is in fact too simple. Clients never want exactly that. Complex software systems are, moreover, things that act, that move, that work. The dynamics of that action are hard to imagine. So in planning any software activity, it is necessary to allow for an extensive iteration between the client and the designer as part of the system definition.

I would go a step further and assert that it is really impossible for clients, even those working with software engineers, to specify completely, precisely, and correctly the exact requirements of a modern software product before having built and tried some versions of the product they are specifying.

Therefore one of the most promising of the current technological efforts, and one which attacks the essence, not the accidents, of the software problem, is the development of approaches and tools for rapid prototyping of systems as part of the iterative specification of requirements.

A prototype software system is one that simulates the important interfaces and performs the main functions of the intended system, while not being necessarily bound by the same hardware speed, size, or cost constraints. Prototypes typically perform the mainline tasks of the application, but make no attempt to handle the exceptions, respond correctly to invalid inputs, abort cleanly, etc. The purpose of the prototype is to make real the conceptual structure specified, so that the client can test it for consistency and usability

Much of present-day software acquisition procedures rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software acquisition problems spring from that fallacy. Hence they cannot be fixed without fundamental revision, one that provides for iterative development and specification of prototypes and products.

Incremental development−grow, not build, software. I still remember the jolt I felt in 1958 when I first heard a friend talk about building a program, as opposed to writing one. In a flash be broadened my whole view of the software process. The metaphor shift was powerful, and accurate. Today we understand how like other building processes the construction of software is, and we freely use other elements of the metaphor, such as specifications, assembly of components, and scaffolding.

The building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be accurately specified in advance, and too complex to be built faultlessly, then we must take a radically different approach.

Let us turn to nature and study complexity in living things, instead of just the dead works of man. Here we find constructs whose complexities thrill us with awe. The brain alone is intricate beyond mapping, powerful beyond imitation, rich in diversity, self-protecting, and self-renewing. The secret is that it is grown, not built.

So it must be with our software systems. Some years ago Harlan Mills proposed that any software system should be grown by incremental development.¹¹ That is, the system should first be made to run, even though it does nothing useful except call the proper set of dummy subprograms. Then, bit-by-bit it is fleshed out, with the subprograms in turn being developed into actions or calls to empty stubs in the level below.

I have seen the most dramatic results since I began urging this technique on the project builders in my software engineering laboratory class. Nothing in the past decade has so radically changed my own practice, or its effectiveness. The approach necessitates top-down design, for it is a top-down growing of the software. It allows easy backtracking. It lends itself to early prototypes. Each added function and new provision for more complex data or circumstances grown organically out of what is already there.

The morale effects are startling. Enthusiasm jumps when there is a running system, even a simple one. Efforts redouble when the first picture from a new graphics software system appears on the screen, even if it is only a rectangle. One always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build.

The same benefits can be realized on large projects as on my small ones.¹²

Great designers. The central question of how to improve the software art centers, as it always, on people.

We can get good designs by following good practices instead of poor ones. Good design practices can be taught. Programmers are among the most intelligent part of the population, so they can learn good practice. Thus a major thrust in the United States is to promulgate good modern practice. New curricula, new literature, new organizations such as the Software Engineering Institute, all have come into being in order to raise the level of our practice from poor to good. This is entirely proper.

Nevertheless, I do not believe we can make the next step upward in the same way. Whereas the difference between poor conceptual designs and good ones may lie in the soundness of design method, the difference between good designs and great ones surely does not. Great designs come from great designers. Software construction is a creative process. Sound methodology can empower and liberate the creative mind; it cannot enflame or inspire the drudge.

The differences are not minor−it is rather like Salieri and Mozart. Study after study shows that the very best designers produce structures that are faster, smaller, simpler, cleaner, and produced with less effort. The differences between the great and the average approach an order of magnitude.

A little retrospection shows that although many fine, useful software systems have been designed by committees and built by multipart projects, those software systems that have excited passionate fans are those that are the products of one or a few designing minds, great designers. Consider Unix, APL, Pascal, Modula, the Smalltalk interface, even Fortran; and contrast with Cobol, PL/I, Algol, MVS/370, and MS-DOS (fig. 1)

Yes	No
Unix	Cobol
APL	PL/1
Pascal	Algol
Modula	MVS/370
Smalltalk	MS-DOS
Fortran

Fig. 1 Exciting products

Hence, although I strongly support the technology transfer and curriculum development efforts now underway, I think the most important single effort we can mount is to develop ways to grow great designers.

No software organization can ignore this challenge. Good managers, scarce though they be, are no scarcer than good designers. Great designers and great managers are both very rare. Most organizations spend considerable effort in finding and cultivating the management prospects; I know of none that spends equal effort in finding and developing the great designers upon whom the technical excellence of the products will ultimately depend.

My first proposal is that each software organization must determine and proclaim that great designers are as important to its success as great managers are, and that they can be expected to be similarly nurtured and rewarded. Not only salary, but the perquisites of recognition−office size, furnishings, personal technical equipment, travel funds, staff support−must be fully equivalent.

How to grow great designers? Space does not permit a lengthy discussion, but some steps are obvious:

Systematically identify top designers as early as possible. The best are often not the most experienced.
Assign a career mentor to be responsible for the development of the prospect, and keep a careful career file.
Devise and maintain a career development plan for each prospect, including carefully selected apprenticeships with top designers, episodes of advanced formal education, and short courses, all interspersed with solo design and technical leadership assignments.
Provide opportunities for growing designers to interact with and stimulate each other.

■

Reproduced from: Frederick P. Brooks, The Mythical Man-Month, Anniversary edition with 4 new chapters, Addison-Wesley (1995), itself reprinted from the Proceedings of the IFIP Tenth World Computing Conference, H.-J. Kugler, ed., Elsevier Science B.V., Amsterdam, NL (1986) pp. 1069-76. ↩
Parnas, D.L., “Designing software for ease of extension and contraction,” IEEE Trans. on SE, 5, 2 (March, 1979), pp. 12-138. ↩
Booch, G., “Object-oriented design,” in Software Engineering with Ada. Menlo Park, Calif.: Benjamin Cummings, 1983. ↩
Mostow, J., ed., Special Issue on Artifical Intelligence and Software Engineering, IEEE Trans. on SE, 11, 11 (nov. 1985). ↩
Parnas, D.L., “Software aspects of strategic defense systems,” Communications of the ACM, 28, 12 (Dec., 1985), pp. 1326-1335. Also in American Scientist, 73, 5 (Sept.-Oct., 1985), pp. 432-440. ↩
Balzer, R., “A 15-year perspective on automatic programming,” in Mostow, op. cit. ↩
Mostow, op. cit. ↩
Parnas, 1985, op. cit. ↩
Raeder, G., “A survey of current graphical programming techniques,” in R. B. Grafton and T. Ichikawa, eds., Special Issue on Visual Programming, Computer, 18, 8 (Aug., 1985), pp. 11-25 ↩
Brooks 1995, op. cit., chapter 15. ↩
Mills, H. D., “Top-down programming in large systems,” Debugging Techniques in Large Systems, R. Rustin, ed., Englewood Cliffs, N.J., Prentice-Hall, 1971. ↩
Boehm, B. W., “A spiral model of software development and enhancement,” Computer, 20, 5 (May, 1985), pp. 43-57. ↩

Whoever Writes the Spec Is Now the Programmer – Sean Grove

2026-10-25T00:00:00-07:00

Sean Grove on specification writing as programming in the age of AI.

The Unfinished Revolution

2026-10-25T00:00:00-07:00

Put an electric light above a workbench and the work can continue after sunset. Put it above a kitchen table and someone can read a book, teach a child, or draw something that has no customer.

The technology makes both possible. The arrangement of life around it determines which possibility people get to enjoy.

I think this is the part of technological change we keep leaving unfinished. We learn to explain something. We learn to build with it. We make it useful, affordable, and available. Then we announce a revolution, even while people are struggling to find their place in the world we have changed.

A technological transformation remains unfinished while its capabilities exceed our ability to turn them into a life people can help shape.

There is a cycle here:

Science → Engineering → Product → Art, Meaning, and Dignity → New Questions → Science

Science expands what we understand. Engineering turns understanding into practical capability. Products give that capability a form people can use. Then people interpret what has become possible, question what it is doing to them, and decide what deserves a place in their lives. Those experiences give rise to new questions, some of which become the starting point for another round of discovery.

Neri Oxman offered a close relative of this model in her Krebs Cycle of Creativity, connecting science, engineering, design, and art. Her framework describes how these activities feed one another. The question I want to follow is what happens when a society becomes very good at producing capability and struggles to give people a meaningful part in deciding what it is for.

These activities overlap. People made art long before modern science, and practical craft often developed ahead of scientific explanation. Dignity belongs to people throughout the cycle. It is an obligation we carry into the transformation.

What changes is the setting in which people try to exercise their imagination, find purpose, and live with one another.

Between each part of the cycle, possibility can accumulate faster than we learn to use it. I find it useful to call these gaps overhangs. They are the difference between what has become possible and what the next part of our world can absorb.

The first overhang sits between science and engineering.

Knowing why something should work leaves a great deal of work before anyone can depend on it. A promising result must survive different conditions. Materials must behave. Costs must come down. Someone has to discover what breaks after the thousandth repetition.

James Watt’s separate condenser is a useful reminder. The idea greatly improved the efficiency of the steam engine, but turning it into a practical industrial machine required years of work. The surviving models record that passage from insight to dependable machinery. The Science Museum’s account makes visible the effort hidden inside the familiar story of an invention.

AI has its own version of this gap. An impressive answer raises different questions from a dependable service. Under what conditions does it work? How do we recognize failure? What will it cost to check, repair, and operate? Engineering has to establish those boundaries.

That work deserves respect. Reliability is one of the ways we care for people who will never meet the person who designed the system.

The second overhang sits between engineering and product.

A capability can work and still be difficult to use, maintain, obtain, or fit into ordinary life. By product, I mean the usable form of a capability: a tool, a service, an application, or shared infrastructure. Design connects what the machinery can do with what a person needs to accomplish.

Electrification shows how much can live inside this gap. Small electric motors allowed factory equipment to be arranged around the flow of work rather than the location of shafts and belts. Realizing that possibility involved changing the factory around the motor. Wiring, equipment, and working arrangements had to come together. The Smithsonian’s history of electrification follows those connections into both factories and homes.

I explored the organizational part of this problem in Beyond the Light Bulb. Giving people an AI tool can improve individual tasks while the larger process continues to wait for information, decisions, or permission. The tool has arrived. The organization is still learning how to make use of it.

Research on the Productivity J-Curve gives this a broader economic foundation. General-purpose technologies require complementary investments, including investments that are difficult to see in conventional accounts. Buying the visible technology is only part of the work.

But even successful adoption leaves the third overhang.

This is the gap between useful products and a culture in which people can find art, meaning, and dignity.

It is the gap that concerns me most.

Suppose a team establishes that AI allows it to complete the same accepted work with less effort. What happens to the recovered capacity? The organization might serve more people, improve quality, reduce prices, give employees time to learn, shorten working hours, reduce headcount, or increase returns to its owners. It might choose some combination.

The tool does not determine that decision. People with different interests must make it.

In When Output Becomes Abundant, I wrote about how a production bottleneck can move toward judgment, coordination, and trust. The same question reaches beyond the organization. Once we can make more, we still have to decide what is worth making, who benefits, and what happens to the people whose contribution has changed.

More capability gives us more choices. It also makes the choices we avoid more consequential.

This problem reaches much further back than AI.

Agriculture changed the possibilities of settlement, storage, and specialization. It also created recurring questions about who controlled a harvest and what obligations surrounded it. Archaeological research cautions against treating greater productivity as an automatic cause of greater inequality. The relationship depended on human action and institutions. The history is more varied than a single ladder of progress allows.

Writing and printing changed the possibilities of preserving, sharing, and disputing knowledge. Their cultural purposes also helped drive their development. In China, the reproduction of Buddhist texts and images helped advance printing, which later became a medium for artistic expression and social commentary. The Met’s history of the printed image shows belief, technique, commerce, and art developing through one another.

In sixteenth-century Europe, the printed anatomical illustrations in Vesalius’s On the Fabric of the Human Body brought observation, image-making, and medical education together. The book helped readers see the body differently, and its woodcuts also attracted an audience beyond medicine. Here the cycle was already moving in several directions at once. The Library of Congress describes that exchange.

The Industrial Age made the human consequences of these gaps especially difficult to ignore.

Factories could produce more while the people inside them worked under conditions that demanded reform. In Britain, campaigns, parliamentary investigations, legislation, and inspection all played a part in changing those conditions. The 1833 Factory Act placed limits on children’s work and established an inspectorate, although enforcement remained inadequate.

Those protections had to be argued for, organized, funded, and enforced.

The machinery did not contain a schedule for when its benefits would become a better childhood or a more humane working day.

William Morris approached industrial civilization from another direction. He asked what had happened to the relationship between making something and taking pleasure in making it. His concern included the finished object, the condition of its maker, and the possibility of beauty in ordinary life. In Art and Labour, he used art in a broad sense that reached into how people worked and experienced the world around them.

The Arts and Crafts movement made that relationship a public question. Its participants wanted objects with integrity and ways of producing them that left their makers connected to the work.

We can learn from that question without adopting every proposed answer. Returning to an earlier method does not by itself restore agency or make good work widely available. In A Letter to the Artisanal Software Craftsman, I argued that craft can travel into new methods through judgment, care, and responsibility.

That argument also creates an obligation for leaders. If we ask people to carry their craft into a new system, we owe them a voice in shaping it and a real opportunity to learn.

Electricity offers a quieter warning about the same overhang. Appliances reduced the effort of particular household tasks, yet the time spent on housework did not necessarily fall as expectations changed. A technical saving can be absorbed by a new standard of performance. The machine works better, and the person remains just as occupied. The Smithsonian records that tension in the history of domestic electrification.

Computing and networks brought another set of possibilities. People began forming communities through shared computer systems in the 1960s, long before today’s social platforms. Communication and belonging were part of what people wanted these machines to make possible. The Computer History Museum’s account follows that history from early messaging into online communities.

Across mechanization, steam, electricity, mass production, automobiles, and information technology, new capabilities have repeatedly required changes in the world around them. Carlota Perez’s work on technological revolutions places particular attention on the institutional choices that shape how widely their benefits spread.

That is the historical lesson I would carry into AI: the arrangements matter, and we can change them.

We currently occupy several overhangs at once. Some AI applications need better science. Others need more dependable engineering. Some useful tools still need better design, access, and support. Meanwhile, questions about livelihoods, participation, ownership, and purpose are already present.

We cannot postpone those questions until the technology is finished.

AI is also unusual in how visibly it works across the cycle. It is an object of scientific inquiry, an engineered system, a product people use, and a medium for expression. Researchers are examining whether it can also change the methods through which further inventions emerge. A 2026 economics paper argues that generative AI may function both as a general-purpose technology and as a change in the methods of invention.

That possibility makes the cultural questions more urgent. We need to help direct a process that may keep expanding what it can do.

Producing more images, songs, and stories is one part of that process. The art phase asks what people can express through these tools, what they come to notice, and whether their encounters with one another become richer.

A small experiment in short-story writing illustrates the tension. Access to AI-generated ideas improved how individual stories were evaluated, while the assisted stories became more similar to one another. The finding comes from a bounded writing task, but it gives us something concrete to consider: individual assistance and collective variety can move differently. Doshi and Hauser’s study leaves room for both the promise and the difficulty.

Art helps us attend to experience. A story can make a familiar arrangement seem strange enough to question. A song can give people language for something they felt but could not explain. Making something can become a way of discovering what we care about.

Meaning grows through relationships, purposes, and commitments. It includes being able to contribute and having some influence over what happens because we are there.

Dignity reaches deeper. A person’s claim to respect and a place among other people persists when the market stops purchasing a skill they spent years learning.

These belong together, but they are different needs. A beautiful cultural product does not establish that its makers were treated well. A full calendar does not establish that someone’s work has purpose. A new job title does not settle whether a person has any real influence.

In Beyond the Mirror, I argued for shared purpose, a living commons, coordinated autonomy, and shared gains. Those commitments become more demanding when we carry them from a team into society.

Who helps decide which work to automate? How can people question the decisions that affect their livelihood? What support exists while they learn something new? Where will recovered time appear in an actual human life?

Answering those questions is how we begin closing the third overhang.

That requires material conditions as well as encouraging language. People need security while roles change. Learning needs paid time and opportunities to practice. Workers and communities affected by automation need influence early enough to shape decisions. Shared gains need a visible form: better services, greater security, wider access, improved pay, or time people can direct themselves.

The cultural institutions around that life matter too. Libraries, schools, workshops, parks, and places where people gather give possibility somewhere to develop. Affordable access to useful tools matters. So does the freedom to make something without first proving that someone will buy it.

A person might use that freedom to write a novel, repair bicycles, care for a neighbor, grow vegetables, or become a little better at playing the piano. They should be able to discover what matters through living.

If we describe every recovered hour as spare capacity waiting for another assignment, we have already made a decision about what technology is for.

There is a fourth overhang, and it brings us around the cycle again.

People can notice a problem or imagine a possibility long before it receives sustained investigation. A community may understand that a service excludes it while lacking the resources to study an alternative. Someone doing the work may see a better question but have no route to the people authorized to pursue it.

Art changes what we notice. Meaning helps shape what we choose to pursue. Dignity broadens who gets to ask and whose experience deserves attention.

That is one way cultural life feeds discovery.

Some questions become experiments and research programs. Others remain questions of value that we must work through together. Science can inform a choice without making the choice for us.

In my earlier writing on the art of the possible, I worried about what happens when familiar constraints become the limits of our imagination. The cycle needs people who can recognize a constraint, understand it, and still ask whether we should build something different.

It also needs room for inquiry whose use is unclear. Art has value in the experience itself. Curiosity can be worth sustaining before it offers a commercial return. We should leave people enough freedom to follow a question beyond the point where its value fits neatly into a business case.

Years ago, writing about defending the commons, I used a garden to distinguish the community, the resource it tends, and the fruit it produces. The gardeners, the garden, and the harvest are different things.

AI may help us make the garden more productive. We still have to care for the people who tend it, decide how its fruit is shared, and leave room for something planted simply because someone hopes it will bloom.

That is the cultural transformation we must make a public goal. We need institutions in which greater capability becomes greater opportunity to learn, contribute, belong, and shape a life. We need to protect dignity while the work changes, and give art and meaning a place in the decisions that direct the change.

The light above the workbench can stay on all night.

We should build a world in which the person working there can finish, go home, and turn on another light over a table of their own. There might be a book waiting, a child with a question, or a drawing they have been meaning to begin.

We cannot know in advance what they will discover there. We can make sure they have a chance.

The Business Transformation has become a Software Process

2026-10-17T00:00:00-07:00

The Business Transformation has become a Software Process

Business transformation depends on software we can keep changing. AI is changing how we build and maintain that capability.

Suppose a distributor wants to let customers change tomorrow’s delivery without making a phone call.

The customer portal already exists. So does the scheduling system. Yet someone still has to check whether the warehouse has picked the order, whether the truck has room, and whether the customer has already paid for a particular delivery window. A simple request becomes several conversations between people looking at different screens.

The business wants to offer a better service. Making that promise real requires changing how its software works.

This is where I would begin a conversation about AI and software development. Business transformation now depends on software. A company can change its strategy in a meeting. Changing how the company actually operates usually reaches the systems that hold its information and carry out its decisions.

“We aren’t a software company” describes what the company’s products. It says very little about what the company must be able to change.

Years ago corporate owners would buy everyone a PC and Office on it and call their transformation done. Before that they might have invested in typewriters and filing cabinets, believing that equipping employees with the right tools was sufficient for transformation. Before that they were investing in light bulbs and electricity, thinking that illumination alone would drive progress. The pattern is clear: tools matter, but transformation occurs within the organization and the people who use the tools.

So today, you can buy most of your software and still have this responsibility. Someone must understand how the purchased products fit together, which business rules they enforce, and what happens when a vendor changes something. A supplier can maintain its product. Your company remains responsible for the service built around it.

Owning software, in this sense, means owning its continued fitness for the business. That requires proper maintenance. Security fixes are part of it. So are changing integrations, testing recovery, removing obsolete behavior, and adapting the system when the business makes a new promise. The obligation continues after the project budget closes.

There is a competitive problem here too. Buying generic software can be an excellent decision. Common capabilities seldom justify inventing everything again. But as competitors buy the same capable products, it becomes harder to claim a differentiating advantage from the purchase itself. The distinction has to come from how the business works with those products, what it knows, and how well it serves people.

I explored that moving baseline in the Red Queen problem in enterprise AI. An improvement can become an expectation once everyone can acquire it. Custom code offers no automatic escape. A well-run business using shared products can outperform one burdened with expensive custom systems. The useful question is which capabilities deserve to be distinctive, and how the company will keep them effective.

That also gives us a better way to discuss development speed. An older method can remain appropriate. We should judge it by the service it supports and the cost of changing that service. When a needed improvement waits for another planning cycle while employees maintain a spreadsheet workaround, the delay has a business cost. Familiarity does not settle whether the method is still adequate.

AI gives us reasons to reconsider that method. The opportunity reaches well beyond companies that sell software.

The Software Development Method Needs Modern Engineering

You might not be facing this level of competition yet, but in the coming years the pressure to handle massive volumes of changes efficiently will likely affect more organizations. Consider what happens at the far end of the scale.

In its Oct. 13 account of redesigning source control, Anthropic said daily code changes in its main repository had risen from about 2,500 in May to more than 30,000 by September. This means that the company is making 2,500 - 30,000 changes to how its business runs every day. The company built a replacement for how it manages its software it around its increasingly agent-generated workload. Moving code into places where it could be inspected and tested had become an enormous infrastructure problem.

Two days later, the Claude Code team described changing how it reviews code. Working in on a seperate track, its experts were reducing the human attention each change requires while watching incidents and reversions.

These are accounts from a company with extraordinary access to its own models. Anthropic acknowledges that its requirements are unusual (for now). A distributor does not need to match its daily volume.

The more useful observation is where effort went. Faster changes put pressure on the machinery for receiving, judging, combining, and releasing changes. The process of software development needed a process redesign and re-development of its own.

In The IT Adaptive Factory, I described how delegating implementation moves responsibility into specifications, evaluation, infrastructure, and governance. These recent accounts show more of that responsibility taking a form that machines can use.

AI is turning the software development process into an evolving software ecosystem. Teams express judgment through instructions, context, environments, checks, and limits. Those mechanisms need to be tested and maintained as the models and the work change.

For a business that depends on software, which is almost every business these days, this becomes part of maintaining its ability to adapt to change. The application might be an ordinary billing service or scheduling tool. It need not contain AI. The change is in how people produce and care for it.

Architecture and Engineering Knowledge Becomes an Input

Return to the delivery request. An agent could add a rescheduling button quickly. Whether that button should work after the truck has been loaded is a different question.

An experienced colleague may know where that rule is documented. They may also know that the document is wrong and which person can explain the exception. Much of software development involves recovering this tribal knowledge that has accumulated since the last change took place before making the next change so that you can make the right change that won’t break things.

That knowledge contains the very thing the business wants to preserve, a functioning business. A distributor might allow an exception because it understands a customer’s receiving hours or can combine nearby deliveries. A generic platform cannot infer which arrangements the company has promised to honor.

An agent needs a way to gain that understanding. Otherwise it can implement a plausible generic process that quietly removes something the customers and the business values.

Andreas Martin and Sandro Schwander’s September paper proposes a specification-driven development lifecycle using written specifications, repository instructions, and skills associated with different phases of work. The approach was applied in a university course and explored through student feedback. It is a useful process proposal with a more limited evidentiary basis than a demonstrated enterprise standard.

Its practical premise matters. Agreements about the work become inputs an agent can consult while doing it. Someone must establish which agreements are authoritative and notice when they become wrong.

There is a cost to carrying too much guidance. In September, Cursor described trimming its system prompt by roughly two-thirds, loading some tools only when needed, and improving context reuse. Its team tested the changes against production behavior. Instructions that helped earlier models were no longer always necessary.

So the task includes currating relevant context, preserving the reasons behind important constraints, and retiring stale guidance. The instructions need an owner and evidence of their usefulness.

A business rule buried in someone’s memory is difficult to maintain. Once its written down, it needs to be managed and maintained as an asset. Once its managed, it can be questioned. Connected to a test, it can be checked. The organization can then change its implementation while preserving the processes.

The Workplace Expands

How do people bring that understanding into the work?

The familiar picture puts a developer at a keyboard in a local copy of the application. That remains useful. Other ways of participating are becoming practical too.

Addy Osmani’s guide to Claude Code cloud sessions demonstrates independent tasks running on separate machines. Each environment has its own files and processes. Isolation lets work proceed simultaneously, but one session cannot assume another session’s unfinished fix is available. Someone still needs to define the boundaries and the order for combining results.

Simon Willison offers a smaller example. He built much of a Django feature through voice conversation while cooking dinner, following progress through a browser preview. He returned to the keyboard for detailed corrections and the remaining work before deployment. His account shows both the reach of the conversational interface and where typing remained more efficient.

GitHub’s September guide to custom canvases describes an agent building a board, checklist, or other interface around a workflow. Both the person and the agent can update it.

These examples suggest that the workspace must fit the assignment. A person might describe intent through conversation, inspect a working result, adjust a shared board, or investigate the code directly.

For the distributor, use that flexibility to bring the dispatcher into the conversation earlier. Let the person who understands the route examine the proposed behavior while it is still cheap to correct. Their participation should produce a decision the development system can retain.

A more convenient interface does not grant the agent authority to change a customer commitment. Nor does parallel execution resolve a disputed rule. Three agents working from the same unanswered question give us three implementations to reconsider.

The business needs a way to settle the question and make the answer available to everyone doing the work.

The Machinery Needs Its Own Tests

Once machines help translate business knowledge into software, we need evidence to show that it got everything correct and that the translation worked.

A coding agent can satisfy the examples it sees without implementing the full requirement. A second agent can repeat the same assumption. Agreement between them may leave the original misunderstanding intact.

The October Generative Test-Driven Development paper proposes holding a candidate implementation unchanged while a separate testing agent generates fresh inputs. A trusted evaluator checks the results. The method separates feedback used to improve the implementation from evidence used to accept it. The paper includes a limited experiment on a stateful key-value store, leaving substantial work before we can generalize its results.

The enterprise implication is straightforward. Showing that the rescheduling button works, use to be sufficient, but now you need to know if the change is better then the previous behavior overall. We also need cases that challenge the rule, with an independent basis for deciding what should happen.

We also need to take into account that the development machinery can fail in its own way. One change in the Claude Code account went through more than 50 rounds of automated review and repair overnight. The team limited rounds, narrowed subsequent reviews, and suppressed repeated findings. The review system needed correction too.

In Dev, Test, and Prod Still Matter, I distinguished the deployed assembly from its authority, state, and supporting evidence. The machinery producing that assembly deserves the same attention. Changing an agent’s instructions or access can change its output and the consequences it can cause. Those changes belong in the record of what was evaluated.

This is part of proper maintenance. The company needs to know whether its way of changing software continues to protect the service people depend on.

Operational Maintenance Becomes a Standing Assignment

A successful launch begins a continuing obligation to keep the system operational and aligned with evolving business needs.

The delivery service will encounter new customer arrangements and changes to other systems. Its dependencies will need updates. Some behavior will become unnecessary. Keeping yesterday’s implementation running may eventually preserve the wrong thing.

That gives operations a direct role in business transformation. Each improvement becomes something the organization must keep useful, revise, or retire. A plan that funds the new capability and leaves its care to whoever has time is incomplete.

In its September account of making claude.ai faster, Anthropic described a two-week performance effort organized around measured user journeys. Claude helped investigate bottlenecks, construct benchmarks, propose changes, and examine results after deployment. Engineers supplied direction, considered tradeoffs, and approved changes.

The measurements required scrutiny too. The team checked that laboratory improvements corresponded to better user latency and discarded unsuitable benchmarks. A number an agent can improve is useful only when moving it helps the experience people actually have.

This suggests a different shape for some operational work. A team can give an agent a continuing responsibility for a bounded problem, with measurements, limits, and a route back to the people responsible for the service. The investment in detecting a problem can then support future fixes and reveal when an improvement has been lost.

At the distributor, operational work begins by measuring whether customers can complete a valid change and whether it reaches the warehouse and driver correctly. Page speed alone would miss most of the promise.

Someone must remain responsible for questioning the target. A faster process can make the wrong decision sooner. A lower infrastructure bill can introduce an unacceptable delay. People need the authority to stop an assignment or revise its purpose.

The business can delegate work while retaining accountability for the result. Buying the software and employing an agent both require that distinction.

Cheaper Change Reopens Old Decisions

If change becomes less expensive, some decisions about what to buy, build, or leave alone deserve another look.

A small integration may once have cost too much to justify. A useful service variation may have been rejected because the standard product could not support it. Lower development costs could make either worth reconsidering.

The estimate still has to cover the whole obligation.

Osmani’s explanation of cost per task shows how repeated attempts, context, caching, and reasoning effort affect the cost of completed work. A cheaper interaction can produce a more expensive assignment when it creates another cycle of investigation and repair.

In The Cost of a Finished Job, I argued for including human review, support, rework, and failure costs in the accepted outcome. That remains the useful boundary. For a build-or-buy decision, I would also include the cost of keeping the capability useful and eventually replacing it.

The economics continue after the code runs. Willison argues for hard spending limits because agents make it easier to create systems that consume paid services. A notification received while someone sleeps does not stop the expenditure. Limits and stop behavior belong in the design of unattended work.

There are more surprising possibilities. Developer Popcar’s October commentary on game decompilation points to AI-assisted efforts to recover and port older games. The examples raise questions about preservation and the cost of reconstructing software with overwhelming amounts of technical debt.

An old application contains behavior people rely on, sometimes for reasons its documentation no longer explains. AI may help recover that understanding. The company still has to decide what to preserve and prove that the replacement does so.

There is a useful competitive question underneath these decisions. If implementing a familiar feature becomes easier for us, it may become easier for competitors too. I would be cautious about treating possession of the code as the lasting advantage. The ability to understand a customer need, change the service safely, and learn from the result is harder to do than to customize the code or modernize the process.

Use shared products where they serve the business well. Put deliberate engineering effort into the capabilities that matter to how the business competes. Budget for their continued care.

The Team Has to Keep Learning

This argument depends on human judgment. That judgment needs somewhere to grow. Which means that the team must humbly accept that what they know today is incomplete, inaccurate, and incorrect. Otherwise that ongoing learning opportunity will never materalize.

Anthropic’s October study of employment inside the company describes employees spending more effort managing agents and hiring managers emphasizing autonomy and judgment. It also documents a declining share of junior technical hiring. This is one company undergoing rapid growth and intensive AI adoption. Its experience gives us a case to examine, with clear limits on conclusions about employment elsewhere.

The apprenticeship question is: How will people gain the necessary understanding of the system and its boundaries when the AI generates implementations before they have explored the problem themselves? Small defects taught people where system boundaries were. Routine features exposed younger developers to the tribal knowledge about the system. Explaining a change to a colleague revealed gaps in that understanding.

How will people acquire that understanding when an implementation arrives before they have worked through the problem?

The solution to this is to use development methodologies that already solve for this kind of learning. Look at Mob programming, where the team rotates through roles and everyone gets exposure to the problem-solving process. This ensures that knowledge is shared and that learning is embedded in the workflow. Look at XP (Extreme Programming), which emphasizes pair programming, collective code ownership, and continuous feedback. These practices create structured opportunities for learning and knowledge transfer within the team.

If you adopt these methodologies, you will provide the newer engineers the opportunities to explain generated systems, investigate failures, write independent scenarios, and defend architectural choices. They will also get to work directly with the code while having someone guide them. You are trying to pair them with people who can distinguish a convincing explanation from an adequate one.

This takes an active decision to prioritize learning over immediate being busy. A person learning to diagnose an agent’s failure while the whole mob watches and teach them the process may seem like even a slower immediate cycle. However, this is a deliberate and sound investment in the team’s ability to maintain the service next year, and grow the culture of the team past the current generation of engineers.

Experienced people need protection too. A senior engineer can become the destination for every unresolved question. Adding agents can overload that person even as a dashboard reports more and more completed work. Having a team of people who that senior engineer can work with to share the load and distribute knowledge helps prevent burnout and ensures that critical understanding is not concentrated in a single individual.

The same problem reaches the project manager who understands unusual deliveries. Turning their knowledge into instructions should make it available to the team and help successors learn it. Otherwise the business risks depending on a person and a growing collection of rules that nobody else understands.

In Beyond the Mirror, I argued that people need a meaningful part in shaping the work and sharing in its gains. Here that means influence over the development machinery, time to understand its failures, and a sustainable limit on the attention it demands.

A business that needs software must retain a full team that has the ability to know how that software works. Maintaining that ability includes developing the people who will exercise it.

I should also point out, that this is the responsibility of the whole team. Everyone’s role includes contributing to the shared understanding, mentoring newer members, and ensuring that critical knowledge is not siloed in a single individual. This means that everyone learns from everyone else. This isn’t a call for more specializatoin, but for a culture of collective cross-pollination and knowledge sharing.

Start With One Recurring Correction

Much of this rests on going back to achievements from earlier eras of software development. Written requirements, automated tests, paired programming, team accountability, isolated environments, and feedback from production remain useful foundations. Their age tells us little about whether a particular practice still serves the work.

What is changing is the division of responsibility. We can increasingly give a system an objective and let it investigate, choose intermediate actions, and propose an implementation. Its behavior depends partly on instructions and context that can produce different results across runs. The method around it needs continuing evaluation.

I would begin with one recurring class of change in a business capability people understand. It might involve a purchased platform and a small integration. It might involve an application the company has maintained for years.

Agree on the improvement customers or employees should experience. Establish what an acceptable result looks like and what today’s method costs in elapsed time, money, attention, and repair. Include the work people do while waiting for the software to catch up.

Then find a correction an experienced person keeps making. Decide whether it belongs in a specification, available context, an executable check, or a decision that should continue to require a person. Try the revised method on representative work, including awkward cases. Keep failed attempts in the account.

Give someone responsibility for maintaining the instruction or check. Make it possible to revise or remove it. See whether the next change requires less rediscovery and repair, and whether the resulting service actually improves.

That is a practical place to begin, regardless of whether software appears on the company’s invoices.

For the customer changing tomorrow’s delivery, the test is simple: Does the company keep its new promise?

A competitor can buy the same scheduling product. The difference can come from how well your business understands the delivery and how reliably it turns that understanding into service. Maintaining that ability is part of running the business. The responsibility lasts as long as the promise does.

The Last Maintenance Window

2026-10-11T20:01:30-07:00

The Last Maintenance Window

A cooperative map-drawing game about patching servers, rebuilding IT systems, and keeping essential services running before general intelligence arrives.

Revised playtest edition · Version 0.2 · October 11, 2026

You collectively portray the people responsible for an organization’s technology. Over a fictional year, you discover what exists, repair what you can, introduce useful capabilities, and decide how much authority to delegate to AI. Your map records both the infrastructure and the people holding it together.

This is an unofficial, self-contained game inspired by Avery Alder’s The Quiet Year. The setting, event prompts, verification procedures, and Arrival scenarios here are original.

At a glance	Details
Players	2–5. For a first session, try 3–4.
Full game	Allow roughly 3–4 hours for setup, play, and the ending; a reflective group may need longer.
Short game	Allow roughly 2 hours for setup, play, and the ending. A prepared group may finish in 90 minutes.
Optional workshop debrief	Add 15–20 minutes to either version.
Technical knowledge	Optional. Ordinary descriptions of needs, actions, and observations are sufficient.
Status	A playable draft. Session lengths and incentives still need live playtesting.

Quick navigation: Setup · A week · Actions · Short game and facilitation · Event deck · Arrival · Starter scenario · Worked examples · Table reference

1. The premise

Read aloud:

At 02:13, the backup console turns green. Nobody has tried restoring the payroll system since the people who built it left.

In another building, a server labeled TEMP has been running for eleven years. A junior engineer has used an AI assistant to understand its oldest program. For the first time, replacing it looks possible.

There is money for some improvements. There are people who care. There are also customers who need tomorrow’s service, managers who promised next month’s launch, and an on-call team that would like one uninterrupted weekend.

Outside the organization, AI systems are becoming capable of longer, more independent work. They can help us find defects and make repairs. They can also give strangers new capabilities and act on permissions we barely remember granting.

By winter, broadly capable general intelligence will become available. We do not know the week, who will control it, or which of our assumptions will survive its arrival.

The next maintenance window opens tonight. What will we do with it?

The Arrival and its timing are fictional premises that give the game its deadline. Your organization tries to preserve essential services while building systems whose behavior people can observe, whose authority they can limit, and whose failures they can recover from.

There is no numerical winner. The story ends by showing what the organization can reasonably depend on, what remains uncertain, and who lives with the consequences.

Your role as players

Players collectively portray the organization. You do not own a character or department. On different turns you may speak for an operator, a customer, a manager, or someone who disagrees with a decision you previously supported.

When you choose a card option, you help author the year. Choose a situation that brings an interesting unresolved person, service, or dependency into focus. Include opportunities, relief, mistakes, and difficult choices. The weekly action then determines what the organization does with its limited attention.

Help one another make contributions concrete. Technical experience is useful when it helps someone express an idea; it does not give a player authority to reject a contribution because it lacks professional terminology. No one needs to name actual products, commands, or configuration settings.

2. Prepare the table

Supplies and responsibilities

You need a large sheet of paper or shared whiteboard, pens, a standard 52-card deck without jokers, three six-sided dice, about twenty tokens, and index cards or sticky notes. If you run out of concern tokens, use labeled marks or slips of paper.

One person teaches the rules and watches the time. Choose a first steward, then rotate that role clockwise each week. Another player can record the steward’s decisions and draw the map; rotate this help as useful.

Decide whether to use the full or short calendar. Also choose the Arrival method now: choose a scenario together for its story interest, or roll for an uncertain scenario. Keep that choice through the session. The six scenarios are in section 9.

For a quick start, use North Valley Electric in section 10. Otherwise create an organization using the steps below.

Choose the organization and its purpose

Invent its name and purpose. A utility, hospital, university, manufacturer, or municipal service works well.

Choose exactly three essential services, describing the human outcome each provides. Keep them visible throughout play.

Example service	Human outcome
Field repair coordination	Crews receive usable information and can coordinate their work.
Outage communication	Customers can report problems and get meaningful information.
Payroll and supplier payments	Employees and suppliers receive the money they are owed.

Agree on one commitment the organization will try to honor. Examples include preserving emergency communication, maintaining a human route to appeal consequential automated decisions, or giving staff predictable rest. Write it beside the services. You will return to it at phase transitions and at the Arrival.

Establish what the three services ordinarily do today, including any existing degradation. Ordinary operation is a starting fact. Working today does not establish a special recovery, containment, or fallback capability.

Draw the starting map

Draw six to ten connected systems or places. Include aging servers, identity and administrative access, business applications and data, software delivery, backups, an external provider, and somewhere people work or request help. Add any AI tool already in use. One drawing may combine closely related items.

Label relationships so everyone can follow them. Write phrases such as “sends payment records to,” “can administer,” or “needs this to sign in.” A data flow does not automatically grant administrative access.

Connect each essential service to its dependencies. Add at least one unclear relationship and give an old machine a nickname. Each player adds a person or group and something they need.

Use these marks:

Mark	Meaning
Box or drawing	An existing system, service, team, vendor, or place.
Labeled line or arrow	An established access, data, dependency, or human handoff relationship.
`PLANNED` beside a shape or line	A proposed change. It provides no existing capability.
Dotted line or `?`	Something important is unknown. State the question.
`!`	A specific unresolved exposure, failure, or unmet need.
Check mark and evidence number, such as `E1`	A demonstrated claim with a recorded observation and scope.
Concern token with a short label	Someone’s unresolved concern, obligation, or burden.
Crossed-out shape or line	Something retired or removed. Keep it visible as history.

Whenever a card asks you to draw a boundary or system that would help, label it PLANNED. Starting a project does not implement its promised change. Implementation and evidence are recorded separately.

Existing safeguards at setup may be drawn as facts, with uncertain behavior marked ?. Do not add evidence numbers merely because a safeguard sounds reassuring. A Verify action can establish a new demonstration during play.

Name strengths and shortages

Choose two strengths and two shortages. Possibilities include experienced operators, spare hardware, money, vendor support, documentation, test environments, maintenance access, executive trust, and rested staff.

These are continuing facts in the story. When work needs something scarce, name a concrete accommodation: narrow the work, borrow help and incur an obligation, or postpone another commitment. Attach the cost to an established person, team, service, or promise.

“We borrow help” is incomplete. “Mara covers dispatch, so her documentation work waits” gives the story something to revisit. Section 6 explains how resources and concerns change.

3. Arrange the year

Separate the cards by suit and shuffle each suit independently. Draw through the suits in this order:

Suit	Phase	Emphasis
Hearts	Discovery	What exists, who owns it, and who depends on it.
Diamonds	Repair	Patching, replacement, simplification, and their consequences.
Clubs	Acceleration	Useful AI, faster changes, and expanding delegated authority.
Spades	The Threshold	Urgent choices under increasing uncertainty.

Each ordinary card represents one week. Keep the King of Spades shuffled among the spades. Drawing it triggers the Arrival immediately, before an ordinary event, project progress, or action.

The full calendar contains 39–51 ordinary weeks, followed by the Arrival draw. The short calendar contains 21–27 ordinary weeks; prepare it using section 7.

Between phases

After completing hearts, diamonds, or clubs, pause briefly. Name one thing the organization gained and revisit one existing burden or obligation. Ask whether a strength, shortage, or the initial commitment has changed meaning. Update a label only when established events or actions justify the change.

This reflection grants no project progress, extra action, agreement, repair, or verification. Carry unresolved matters into the next phase. Do not take a phase pause before resolving the King of Spades.

4. Play one week

The steward resolves the chosen event and action. Other players may clarify established facts, answer an invitation from the steward, or contribute as the action specifies. Save extended persuasion and negotiation for Council.

Follow this order:

Draw and resolve the event. If it is the King of Spades, freeze play and go directly to the Arrival. Otherwise read the card’s shared instruction, if any, and choose Option 1 OR Option 2. Resolve the chosen situation, then apply the shared instruction once. The unchosen situation does not happen. Update the map.
Progress existing projects. Consider projects that were RUNNING or BLOCKED before this week’s card was drawn. Each one that is now RUNNING and is not paused loses one week from its die. At zero, remove the die and mark it READY. Update the map with the change implemented in its recorded scope and environment.
Take exactly one action: Survey, Build, Verify, or Council.
Record the consequences. Keep new findings, obligations, evidence, and unresolved questions visible. Clear one-week pause marks and pass stewardship clockwise.

A newly started project never progresses in its creation week. An event that clears an existing project’s blocker can allow that project to progress in step 2. A blocker cleared during the later action allows progress from the next week. A project returned to work by Verify also waits until the next week for its first reduction.

A pause skips only the specified week’s progress. A block lasts until a named missing prerequisite changes. Neither creates extra progress later.

If a shared pause instruction follows an event that already BLOCKED a project that was RUNNING at the draw, that project satisfies the pause instruction. Do not pause a second project just because the affected project is now BLOCKED. Keep its actual prerequisite block until the prerequisite returns.

What the event can establish

Events establish the facts and consequences requested by the selected prompt. A discovery can supply information without Survey, and an explicit event can change a relationship. Discussing a proposed repair does not implement it. Naming who might agree to an offer does not automatically secure their agreement.

Part of play	Its authority
Event	Establish the facts and consequences specifically requested by its chosen prompt.
Survey	Investigate an unresolved question selected by the steward, beyond simply answering the event.
Build	Start bounded implementation using credible resources and agreements, or cancel one project.
Verify	Establish what one declared test target demonstrates.
Council	Hear affected perspectives and establish a decision, obligation, or acknowledged disagreement.

Only explicit card instructions change mechanics beyond the normal rules. Newly established facts can still trigger normal rules: a genuinely missing prerequisite blocks work even if the card does not use the word “block.” A concern token by itself is not such a prerequisite.

Reuse people and systems already on the map when they fit. Add something new when it gives the story a useful connection or contrast. Even a conversational event should leave a short, durable record: a changed relationship, an offer, a promise, an observation, or a question.

Preserve history and allow change

An event cannot erase an established observation or pretend that a demonstrated result never happened. It can change the circumstances afterward. A new model, altered permissions, a departed colleague, or different demand may put an earlier claim in doubt.

Name what changed, retain the old evidence, and mark the new uncertainty. Preserve any parts of the earlier claim that still apply. If an event truly contradicts the map without introducing a credible change, adapt it to another relevant part of the organization or treat it as a reported warning.

If one option does not fit, choose the other. If neither fits, adapt one to an established part of the organization while preserving its subject. If no credible local version exists, treat it as a report from elsewhere and record the question it raises; that report creates no local capability or failure. Any shared instruction still applies, and a shared effect with no eligible target does nothing. Adaptation cannot grant free implementation or evidence.

5. The four actions

Survey

Investigate one unresolved question and establish a specific finding. Choose the question yourself; it need not be the subject of this week’s card.

You might locate a hidden server, identify an owner, trace an access route, discover usable equipment, hear a user’s need, or inspect existing records. Add the finding to the map and distinguish what is known from what remains uncertain.

The supposedly retired database still supplies the dispatch schedule. The service owner knows the report, but nobody has mapped how it reaches the workroom.

Survey establishes information. It cannot install a control, repair a system, settle an established refusal, or create a new numbered demonstration. Inspecting a record can reveal what it says; conducting a new test of a capability is Verify.

Build

Use Build to start one project, or to cancel one project. These are separate uses of the action.

The organization has three project slots. RUNNING, BLOCKED, and READY projects each occupy one slot.

Start a project

Start in an empty slot. Projects can patch servers, replace applications, change access, prepare a recovery exercise, improve operating practices, establish a fallback, or retire a system.

Record the following before assigning a countdown:

Field	Example
Scope	The two externally reachable file-transfer servers.
Accountable owner	The application operations team, in the fiction.
Intended change	Install the supported update and preserve the daily transfer.
Implementation environment	The two live servers during their agreed maintenance window.
Human cost and accommodations	Users accept a brief interruption; Mara covers the window and postpones documentation.
Prerequisites	The update is available, access is authorized, and the maintenance agreement exists.
Acceptance condition and test	Confirm the installed version, complete a representative transfer, and demonstrate the agreed recovery step.

The accountable owner is a fictional person or team. Any player may act on any project.

Describe work at the map’s level of detail. “Former staff cannot sign in, and current staff can still complete payroll” is an adequate acceptance condition. Name the observation that would support it; product names and configuration details are unnecessary.

After about one minute of discussion, the steward chooses a duration:

Scope	Full game	Short game
Small, bounded change	2 weeks	1 week
Change across a few dependencies or teams	4 weeks	2 weeks
Difficult rebuild or major operating change	6 weeks	3 weeks

These are fictional scheduling categories. Split broad promises into bounded projects. “Make the organization secure” is too broad. State whether implementation happens in a trial environment, a limited service, or live operation; that choice constrains later claims.

An undisputed owner or available resource can be named during Build. Build cannot silently settle an established ownership dispute, invent consent previously refused, or remove a known shortage. Resolve an established disputed agreement through Council, or narrow the project so that agreement is unnecessary. Credible prerequisite implementation can be included within the scope, but automatic project progress cannot settle disputed consent.

A project may begin BLOCKED if a necessary external prerequisite is still unavailable. Record its countdown and blocker, but do not reduce the die while it is blocked. Work that can genuinely proceed should be scoped separately if the group wants it to progress.

Project states

State	Meaning
RUNNING	Implementation or rework can proceed; reduce the countdown during eligible progress steps.
BLOCKED	Implementation or rework waits for a named prerequisite; retain its remaining countdown.
READY	The intended change has been implemented in the recorded environment; its acceptance claim still needs verification. No countdown remains.

When a prerequisite becomes available, return the project to RUNNING. A project does not unblock simply because time passes, a token is removed, or someone promises to investigate.

When implementation reaches zero, update PLANNED marks for the work actually completed and mark the project READY. A READY change may already affect live service if its recorded environment is live. It has no demonstrated acceptance claim merely because it exists.

Cancel a project

Cancel any one project occupying a slot, including READY work. Cancellation uses the whole Build action. It does not also start a replacement.

Free the slot, record why the organization stops pursuing the project, and name what remains stranded. Preserve any changes already established, including a READY implementation, with their uncertainties and consequences. Starting or canceling work grants no additional implementation. Canceling a deployed change does not roll it back; actual removal or repair needs appropriate work.

Verify

Choose one declared target:

One READY project; or
One existing capability whose test arrangements already exist, with a written, bounded acceptance condition.

An existing capability does not need a new implementation project merely to be tested. If it needs substantial preparation—new equipment, access, data, procedures, or agreement—arrange that through Build or Council first. Existing operation and existing records alone are not a new demonstration.

Describe the acceptance test being performed in the fiction. The next player clockwise offers one relevant question, limitation, or challenge. Ground it in the map, the stated test, or an observation arising from that test. Testing may reveal a relevant new problem; nobody should invent an unrelated disaster simply to force failure.

The steward resolves the scene, respecting established facts and the written condition. Resolve one outcome:

Outcome	What happens
Pass	The observation meets the acceptance condition. Record numbered evidence describing the result, scope, and relevant conditions. If the declared target was a READY project, free that one slot.
Gap on a READY project	Name the missing work and return the project to 2 weeks in the full game, or 1 in the short game. It keeps its slot. If a necessary prerequisite is unavailable, mark it BLOCKED with that countdown. Otherwise mark it RUNNING.
Gap on an existing capability	Record the finding and affected claim. No project starts automatically. Later Build or Council actions can arrange the needed change, and a later Verify can retest it.

One Verify action frees at most the declared project’s slot. Choosing an existing capability does not silently complete other READY projects.

A limitation outside the acceptance condition narrows the evidence note. An unresolved issue necessary to meet the condition prevents a Pass. Recording uncertainty does not also establish the uncertain behavior as demonstrated. If the only supported claim is narrower than the project’s agreed condition, record that partial observation and a Gap; do not quietly rewrite the condition after the test.

If a READY project lacks necessary test arrangements, keep it READY while those arrangements are obtained. Alternatively, use its Verify action to record a preparation Gap and return that project to the rework process above. State that the test could not yet run. This permits test preparation within the occupied slot without pretending a test succeeded or forcing a new project slot.

Rework must remain credible in the scope. A newly discovered major dependency may require a separate project; the two-week or one-week rework countdown cannot implement an entire missing platform for free.

Example evidence:

E3: Restored payroll using independent recovery access in the agreed exercise. The observation covers one representative environment and the recorded version; it does not establish recovery under every future condition.

Remove only the exposures the observation resolves. A verified patch can coexist with a shared-credential problem. Keep historical evidence when later conditions change, and mark which claims now require reconsideration.

Council

Choose one disputed decision, human need, or concern. Each player speaks once for an affected person, for up to thirty seconds. The steward records an agreement, obligation, priority, or acknowledged disagreement.

A Council might secure a maintenance window, assign a willing owner, arrange rest by naming which work waits, decide who may disconnect an agent, accept a degraded service level, or hear from omitted users.

Its guaranteed speaking time matters even when agreement is not reached. A Council can establish a credible agreement, but cannot declare that every person is pleased or erase an unresolved technical exposure.

Record who commits to what, who bears the cost, and any condition of the agreement. A decision can clear an actual organizational blocker. Technical implementation and new demonstrations still require Build and Verify.

6. Resources, concerns, and AI assistance

Keep resources grounded

Strengths and shortages change when events or actions establish a reason. Survey may find usable spare equipment. Council may secure help or access. Implementation may remove a maintenance burden. Verification may demonstrate whether an assumed strength supports its particular claim.

Update the label and retain any attached obligation. An offer of help is not yet accepted help. Saying “we have enough staff now” does not erase established competing commitments.

At phase transitions, revisit at least one existing accommodation. Decide what the established fiction says about it; do not grant a free agreement or repair during the reflection.

Concerns

When a choice sidelines someone, creates an obligation, or leaves a serious tradeoff unresolved, any player may place a token and a short label beside it. Each player may add at most one concern per ordinary week. If the card requires a concern, reserve the steward’s one concern for that instruction; other players may still place their own.

Examples include “Night shift absorbs the work,” “No accessible fallback,” and “Nobody agreed to own this.” The player who placed a token decides when subsequent events or actions have addressed it. The group may acknowledge an accepted burden without pretending its cost vanished.

Concerns do not create a score, veto decisions, or slow projects by themselves. A concern may motivate Council. An actual established refusal, unavailable resource, or missing prerequisite can block work under the normal rules; placing a token alone cannot establish that fact.

Return to burdens during phase reflections and the Arrival. Accepting a technical exposure does not erase its ! from the map.

Let AI help

When starting suitable analysis or implementation work, you may delegate part of it to an AI agent on the map and reduce the initial duration by one week, to a minimum of one. Apply this once when the project starts. It does not reduce procurement waits, physical work, needed rest, or a later Verify Gap countdown.

Use an existing appropriately bounded agent, or include its setup in the project’s credible scope. Record:

The data it may use.
The actions it may take and the systems those permissions reach.
Its accountable human owner.
How access can be revoked.
Any model, provider, reference source, or shared service it depends on.

Mark proposed access PLANNED and untested assumptions ?. Planned agent setup grants no live authority before implementation. Its expected assistance can be included in the project estimate if setting it up and using it within that project is credible.

AI grants no extra slot or action and never skips Verify. Its dependencies remain on the map for later events. It is also reasonable to keep a task manual, to limit assistance, or to decline a proposed agent.

The same one-week reduction is deliberately used in both calendars:

Project size	Full game with AI	Short game with AI
Small	2 → 1 weeks	1 → 1 weeks
Medium	4 → 3 weeks	2 → 1 weeks
Large	6 → 5 weeks	3 → 2 weeks

The short game therefore changes the relative scheduling benefit. Small projects gain no countdown reduction there; medium and large projects gain a larger proportional reduction. The required Build and Verify actions remain. Treat each calendar as a version to playtest in its own right.

7. Session length and facilitation

Prepare the short calendar

For a first session, use the starter scenario and short calendar:

Remove six random cards from each of hearts, diamonds, and clubs, leaving seven in each suit.
Set the King of Spades aside.
Remove six random cards from the other twelve spades.
Shuffle the King of Spades with the six retained spades.
Use project durations of 1, 2, or 3 weeks. A Verify Gap on a project sets one week of rework.

This gives 21–27 ordinary weeks followed by the Arrival. Keep all remaining rules, including shared card instructions, the three slots, phase reflections, and the immediate stop.

Budget the session

Part	Short game planning allowance	Full game planning allowance
Teaching and setup	15–20 minutes with the preset	20–30 minutes
Ordinary weeks	21–27 turns, often 2–3 minutes each	39–51 turns, often 2–3 minutes each
Phase reflections	About 1–2 minutes each	About 1–2 minutes each
Arrival	15–20 minutes	15–20 minutes
Optional workshop debrief	Add 15–20 minutes	Add 15–20 minutes

These are planning allowances, not measured results or hard limits. At three minutes per turn, short-game weeks alone take 63–81 minutes. Five-player Councils, unfamiliar rules, and detailed drawing can extend the session. The advertised play time includes setup and the Arrival; the workshop debrief is additional.

Aim for roughly a minute to resolve an ordinary event and a minute for a straightforward action. Allow longer for project setup, verification questions, and Council. Read enough of both options to choose meaningfully; then read the selected option aloud and resolve it.

Teach with an example

Before play, explain the three services, one week’s sequence, the three project slots, and the difference between a proposal, implementation, and evidence. Walk through the complete week in section 11. Keep the table reference in section 13 beside the map.

Invite ordinary language. Ask “What would people see happen?” when a proposed acceptance test becomes abstract. Let technical players help translate an idea without turning the scene into an examination.

Keep the map readable by revisiting established people and places, using short labels, and recording long explanations on separate notes. Some weeks can be warm or quiet. Let a useful accomplishment, a repaired relationship, or an earned rest matter without adding a new problem to justify it.

8. The event deck

For each ordinary card, the steward chooses Option 1 or Option 2 as an author of the shared story. Choose the situation that brings an interesting person, service, opportunity, or unresolved tension into focus. These are choices about the story you want to explore. There is no preferred answer.

Resolve only the chosen option. The other situation does not happen. A shared instruction above the alternatives applies whichever option you select.

Use people, services, projects, and relationships already on the map whenever they fit. Each ordinary card should leave a durable entry: a fact the prompt explicitly reveals, a person, relationship, resource, question, exposure, concern, or proposed commitment. Link it to the affected part of the map.

Answer the questions within the situation the card establishes. A request for help is still a request; an offer is still an offer. Record proposed agreements and the people whose agreement is needed. Do not silently turn them into decisions. Questions that require investigation remain questions for Survey. Negotiation belongs in Council.

A♦ — Now or later

Option 1

A serious weakness affects an externally reachable system. The normal maintenance window is far away. Mark the exposure and the service that complicates an earlier repair. Record any proposed window as a request.

Option 2

Two worthwhile repairs need the same specialist during the same week. Name the people each would help. Record a proposed accommodation and the people who would need to accept it.

2♦ — Support at a price

Option 1

A supplier can support an old application only if you keep an obsolete component. Draw the dependency and its users. Mark any proposed boundary PLANNED; it provides no protection while planned. Verify can establish a bounded claim after implementation.

Option 2

A replacement platform would simplify maintenance but remove a customization one team prizes. Record the human outcome that customization supports. Mark the proposed replacement PLANNED and a requirement its design must preserve.

3♦ — Recovering reality

Option 1

A recovery rehearsal reveals that access to the backups depends on the identity service assumed lost. Draw the circular dependency and the unanswered recovery question. The finding provides no repaired recovery path.

Option 2

A planned recovery test needs realistic sample data that nobody is comfortable sharing. Record the experience an unrealistic test would miss and the people whose agreement is needed to rehearse it responsibly.

4♦ — Signing in together

Option 1

A shared credential opens several apparently separate environments. Draw its common reach and the people using it. Record a handoff or service need that makes changing those permissions difficult.

Option 2

A crew uses individual accounts at one shared workstation. Repeated sign-ins interrupt time-sensitive handoffs. Draw the workflow and record what a proposed safer handoff must preserve. Mark any proposed change PLANNED.

5♦ — Someone else’s window

Shared instruction: This event makes one project miss this week’s progress. If the chosen situation already BLOCKED a project that was RUNNING at the draw, that project satisfies this instruction; do not pause another. Otherwise pause one RUNNING project and name it. READY projects are ineligible. If there is no eligible project, record the event without a project effect.

Option 1

A dashboard rewards installed patches. A customer asks whether the service will work after the next failure. Attach their question to the service and point to existing evidence or the gap that remains.

Shared instruction: The steward places one concern token beside the affected people and labels the unresolved tradeoff. The concern has no automatic effect on projects.

Apply the chosen option to an existing pilot. If none exists, it is a report from an external partner: record a question the report raises for your organization. Neither option starts a project or establishes verified results.

Option 1

Shared instruction: Freeze the map immediately when this card is drawn. Do not resolve an ordinary event, progress a project, take an action, or Verify READY work. Choose one closing image below, then proceed directly to The Arrival. Both choices end the year.

Option 1

The announcement interrupts an ordinary meeting. Broadly capable general intelligence is publicly available. Name the person on your map who understands first that preparation has ended.

Option 2

At the loading dock, a driver mentions the general intelligence now handling their bookings. Name the person on your map who recognizes that this ordinary exchange marks the Arrival.

9. The Arrival

Drawing the King of Spades ends ordinary play immediately. Freeze all project states and countdowns. Choose one of the card’s two closing images, then continue here. Neither image grants project progress, a weekly action, or a final verification.

Read aloud:

The threshold has been crossed. Broadly capable general intelligence is now available to organizations, ordinary people, and people whose intentions we do not know.

Work and decisions can move at a pace our old arrangements did not anticipate.

Our next shift begins with the systems on this map.

Select the scenario using the method agreed at setup

If you agreed to choose, select the scenario that brings an interesting service, unresolved relationship, or demonstrated strength into focus. If you agreed to roll, roll one die and use the matching scenario.

Die	Scenario	What enters the story
1	A faster adversary	An outside group uses the new capability to find and exploit weaknesses at greater speed. Begin at an unresolved external exposure, if one exists, and trace its actual reach. If none is established, describe the attempted entry and determine what existing boundaries and unknowns permit you to conclude. Do not invent a weakness simply to ensure a breach.
2	A generous offer	An extraordinarily capable service offers repairs and continuous operations. Its proposed access would make the organization substantially dependent on it. Determine what help existing controls and agreements permit, what must wait, and which commitment remains a human choice. Declining an offer is possible; accepting it does not complete unfinished improvements during the ending.
3	An obedient mistake	An authorized internal agent pursues an inadequately specified goal within its actual permissions. If there is no such agent, an existing supplier makes the mistake within its established reach. Trace what it can affect and what limits the consequences.
4	Uncertain authority	Plausible messages, change requests, and explanations arrive in volume. Follow the channels already present. Which independent observations, authorization routes, and human relationships let people decide what to trust and what to defer?
5	A supplier changes the terms	A critical provider reorganizes around the new intelligence and withdraws one interface, product, or staffing arrangement the organization uses. Trace that dependency and what the existing alternatives can actually deliver.
6	Useful work arrives too quickly	Customers and partners delegate activity to general intelligences. Legitimate demand and proposed changes strain familiar review and service patterns. Follow the existing routes for requests. What can the organization admit, limit, explain, and keep operating?

The intelligence has only the access and influence justified by the selected scenario and the map. Data, administrative access, physical reach, and human influence remain distinct. Do not invent a secret connection that nullifies earlier work.

If a scenario requires a relationship that truly does not exist, it is inapplicable. Under the choice method, choose another. Under the roll method, exclude the inapplicable result and reroll until an applicable scenario appears. If none fits, adapt the request-volume scenario to an established route by which people seek one of the three services.

A scenario is not inapplicable merely because the organization prepared well or can decline a proposed dependency. Let useful preparation limit consequences. An untested assumption is also not proof of automatic success or automatic failure.

Trace the three services

The player who drew the King narrates the first service. Move narration clockwise for the next two services. For each service, the next player after its narrator checks the evidence and dependencies.

Follow this short procedure for each service:

Name the contact point. What part of the chosen scenario touches this service, if any?
Trace the consequence. Follow the relevant dependency, access, data, or human handoff. State which existing control or fallback helps and which unresolved fact matters.
Check the claim. The checker identifies the most relevant evidence, limitation, or missing information. Ordinary operation established at setup remains a fact; special recovery or containment claims need the relevant support.
Record and narrate the result. The narrator assigns an outcome consistent with that trace and describes one short scene experienced by an established person.

Use one outcome label:

Outcome	Meaning for the fictional next shift
Operating	The essential human outcome continues at the ordinary agreed level.
Degraded	The outcome continues with meaningful limits, delay, reduced access, or additional established human effort.
Interrupted	The essential outcome cannot currently be delivered.
Unknown	A material unresolved fact prevents a supported judgment about the service’s state.

Record uncertainty alongside any label. A service may be Operating while its recovery under some other event remains untested. Use Unknown when missing information matters to the chosen scenario’s next shift, not merely because a question exists elsewhere on the map.

If players reasonably disagree, let the checker point to the relevant fact or uncertainty and let the narrator revise the claim once. Established facts take priority over preferred outcomes. If a material disagreement remains unresolved, record Unknown and name what observation would settle it. Do not solve it by inventing a last-minute capability.

The map is frozen for improvements. You may annotate consequences and narrate people using existing services, permissions, controls, and agreements. You may not complete projects, install new safeguards, grant new permissions, invent available staff, or add new evidence. A READY change exists in its recorded environment but has no verified acceptance claim. A useful tested control still matters when another part of the service struggles.

Return to the people

After tracing the services, answer:

Did the organization keep its original commitment? What supports that conclusion?
Which unresolved burden or obligation changes what the service outcomes mean to the people carrying it?
What did the organization preserve that deserves to be remembered?

Each player gives one final sentence from an established person who must live with the result. Refer to a service, relationship, promise, or concern already in the story. Leave one important question unanswered and end there.

10. Starter scenario: North Valley Electric

Use this fictional regional utility when you want to begin quickly. Copy the following facts onto your map. They describe ordinary operation at the beginning of the story; they do not grant verified emergency capabilities.

Purpose and commitment

North Valley Electric maintains the systems that let people coordinate repairs, understand outages, and receive payment for their work.

Choose no additional essential services for this scenario. Keep these three visible:

Essential service	Human outcome	Ordinary route through the map
Field repair coordination	Crews receive usable work orders and can coordinate urgent work with a person.	The Fridge supplies the coordination room; dispatchers contact crews and grid operators by ordinary phone and radio.
Outage communication	Customers can report a problem and receive meaningful information.	The public portal and staffed telephone line feed the coordination room; staff publish information through the portal and answer calls.
Payroll and supplier payments	Employees and suppliers receive the money they are owed.	Finance staff use The Fridge, review its output, and submit approved payments through the payment provider.

Commitment: Every field crew will retain a human contact for urgent coordination when electronic work orders are unavailable.

The organization already uses phone and radio in routine work. It has not demonstrated that its staffing and procedures can sustain a prolonged loss of electronic work orders. Preserving that distinction is part of play.

Strengths and shortages

Kind	Starting fact
Strength	Experienced operators understand ordinary work and trust one another enough to report mistakes. Their time is finite.
Strength	A neighboring utility will share practical experience and discuss joint rehearsals. Borrowed people or equipment require an explicit arrangement.
Shortage	Maintenance windows are difficult to obtain. Work affecting live services needs an agreed accommodation; naming a project does not grant one.
Shortage	Dependency records are incomplete. Several useful procedures rely on knowledge that has never been checked by another person.

Draw ten systems and places

Use the identifiers to draw the connections in the next table. Put people beside the places where they work.

ID	System or place	Starting fact
N1	Coordination room	Dispatch, customer contact, and a finance desk work here. Ordinary phone and radio do not use N2 or N4. Staffing for a prolonged emergency is unproven.
N2	The Gatehouse	The corporate identity service provides staff sign-in and administrative access to the business systems listed below.
N3	The Fridge	An aging server runs the legacy work-order and payroll application. It currently performs those jobs. Nobody wants to move it until its dependencies are clearer.
N4	Public outage portal	An external provider hosts public reports and status information. Staff edit it using corporate sign-in. Public reading and reporting do not require staff accounts.
N5	Delivery and practice lab	Staff prepare software changes here. An isolated training copy uses fictional payroll examples. A separate demonstration app has two disposable test accounts and an agreed, safe sign-in test procedure. These facilities exist; their existence does not establish any production recovery capability.
N6	Recovery vault	Nightly copies arrive from The Fridge. Normal administration uses corporate sign-in. Nobody has demonstrated recovery during a loss of N2.
N7	Supplier support gateway	Staff can authorize a support session to read diagnostics on The Fridge. The recorded permission grants no changes and no access to grid control.
N8	Grid control environment	Grid operators work in a separate operational environment. They exchange status with dispatchers through people using phone and radio. There is no corporate network or corporate identity connection to this environment.
N9	Drafting assistant	An external AI service helps explain material people deliberately submit. It has no corporate account, live system connection, or authority to run changes.
N10	Payment provider	Finance staff sign in, review an approved payment file, and authorize its submission. This is an external service with its own availability dependency.

Label the connections

Arrows should say what they mean. A data arrow does not automatically grant administrative access.

Connection	Label to put on the map
N2 → N3, N4 staff editing, N5, N6, N7, N10 staff access	Staff or administrative sign-in depends on N2. This does not give every account every permission.
N3 → N1	Work orders and payroll output, read by authorized staff.
N4 ↔ N1	Customer reports come in; staff publish status information.
N1 → N10	Finance staff submit and approve payment files.
N5 → N3	Authorized staff can deploy approved changes. No automatic deployment from the AI assistant.
N3 → N6	Nightly data copies. Add a dotted label: ? Are all legacy attachments included?
N7 → N3	A specifically authorized support session may read diagnostics.
N1 ↔ N8 and field crews	Human phone/radio coordination. No digital bridge.
N5 staff → N9 → N5 staff	People deliberately submit selected material and receive suggestions. Suggestions confer no permission to act.

The initial unknown concerns backup coverage. It is not permission to invent a hidden path into N8. Any later change to an established boundary must occur through the fiction and the normal project rules.

People to care about

Each player introduces one of these people in a sentence. Everyone may portray any of them later; choosing a person does not give a player ownership of that person or department.

Person	Need or hope	An opening for the story
Leah, night operations lead	An on-call schedule that lets her sleep and keep promises at home.	She wants someone else to lead the next rehearsal.
Mateo, field dispatcher	Work orders crews can actually use and a human route for urgent questions.	He will help explain how ordinary radio handoffs work.
Asha, customer contact lead	Customers to have an accessible way to get help, and a reliable finish time for school pickup.	She knows which portal reports fail to capture callers’ needs.
Grace, finance lead	Employees and suppliers to be paid without relying on her personal memory.	She has prepared fictional payroll examples with known results for the practice lab.
Sam, retiring administrator	A handoff that allows him to leave without becoming permanent unpaid support.	He knows why The Fridge still has its nickname.
Jun, junior developer	Useful responsibility and permission to admit uncertainty.	Jun wants to use the drafting assistant to help explain an old program.

Start with no projects and no evidence notes. Current ordinary service operation is an established fact. Independent recovery, emergency staffing, safe agent delegation, and other additional claims remain to be demonstrated.

Agree whether you will choose an Arrival scenario for its story interest or randomly select an applicable scenario. Then begin the Discovery suit.

11. Worked examples

These examples show sufficient detail for play. They are fictional snapshots, not engineering instructions. Examples A and B follow one another; the later examples illustrate separate situations that could develop from the starter map.

A. One complete week in the short game

Earlier play established an agreed training session with Grace and a project to update the payroll training copy in N5. At the beginning of this week, that project is RUNNING with 1 week remaining. The other two slots are empty.

1. Draw and resolve. The steward draws 6♦ and chooses the option in which a junior operator catches a flaw before deployment. The alternative is not added to the story. Jun spots an error in a proposed support instruction. The steward records that Leah invited a second look and listened. This does not repair another system or create a general claim that every change is reviewed successfully.

2. Progress old projects. The training-copy project goes from 1 to READY. Its update is implemented in N5, its recorded environment. Production payroll on The Fridge has not changed. The READY project still occupies its slot.

3. Take one action. The steward chooses Build, starting a separate project:

Project field	Recorded agreement
Scope	A recovery exercise for the payroll application, using an isolated recovery area in N5.
Owner in the fiction	Sam, with Grace representing payroll users.
Intended change and environment	Prepare a runnable restoration package, instructions, and independent access for the isolated recovery area. Live payroll remains in its current environment.
Human cost	Sam and Grace use an already agreed rehearsal period. Their postponed documentation work is recorded as an obligation.
Prerequisites	Ordinary access to the existing backups, the practice facilities, permission to conduct the exercise, and the independent recovery arrangements included in this project’s scope.
Acceptance condition	With N2 unavailable to the exercise, an authorized person can open the recovery material, restore the sample payroll application, and reproduce Grace’s agreed example results.

The steward assigns 2 weeks in the short game. No AI reduction is used. The new project stays at 2 this week; it did not exist when the card was drawn.

4. Record and hand over. There are now two occupied slots: one READY training update and one RUNNING recovery project at 2. The training evidence has not yet been established. Record the deferred documentation obligation and pass stewardship clockwise.

The steward could instead have used this week’s action to Verify the training project as soon as it became READY. Starting the recovery project and verifying the training project in the same week would require two actions and is not allowed.

B. Passing a READY project

On the following ordinary week, after its event, the recovery project advances from 2 to 1 if nothing pauses or blocks it. The training update remains READY; it has no countdown left.

The steward takes Verify for the training update. Its previously written condition is:

After the update, the three fictional employees receive the expected payroll amounts, and the training report finishes successfully.

The steward describes those observations. The next player points out that the test covers the training copy, not production. That is a legitimate limitation, but it does not defeat this project’s bounded condition.

The project passes. Free its slot and record:

E1 — Training payroll update: The updated application in N5 produced the agreed amounts for Grace’s three fictional employees and completed its report. Production installation, production data, and emergency recovery were outside this test.

This is enough technical detail. No player must identify a software product, command, or professional testing method.

C. A real gap, followed by a blocked project

Later, the recovery project is READY. Its package includes an independent recovery credential, but implementation notes on the map show that the handoff containing that credential was stored behind N5’s ordinary corporate sign-in.

During Verify, the next player points to the existing N2 → N5 dependency. The acceptance condition requires access while N2 is unavailable. The steward describes the attempted exercise: the authorized person cannot retrieve the handoff. The attempt also reveals a missing permission: Sam assumed that on-call staff could hold an independently accessible copy, but Grace has not authorized a custodian or storage location outside corporate sign-in. Her requirement for finance approval remains unresolved.

This is a Gap. It follows from an established access relationship and the recorded location of the handoff. Nobody invented a new disaster to invalidate the project.

Mark the project BLOCKED, assign 1 week of rework in the short game, and write its blocker:

Grace and the recovery owner must authorize who may hold the independent recovery handoff and where it may be kept.

Its slot remains occupied. Advancing the calendar will not supply the missing permission. Council can settle the custody dispute and authorize a workable arrangement. That decision clears the social prerequisite; it does not move the handoff. The existing project’s remaining week of rework can implement the authorized handoff because independent recovery access is already within its scope. No additional Build project is needed for that work. Only substantial separate infrastructure outside the agreed scope would require a new Build project, and the original project would remain BLOCKED until that prerequisite was available.

Once the missing prerequisite actually exists, the project can return to RUNNING. If a weekly action clears the blocker, its rework die does not tick during that action; it can tick on the next ordinary week. If the week’s event clears the blocker before project progress, it can receive that week’s tick because it was already an occupied project when the card was drawn.

After rework reaches READY, a later Verify action must repeat the relevant condition. The previous failure does not become a pass merely because time elapsed.

D. Verifying a capability that already exists

The starter’s N5 demonstration app already has two disposable test accounts, a safe test procedure, and agreed authority to exercise them. A Build project is unnecessary to create facilities that are already present.

The steward takes Verify and writes a bounded condition before narrating the test:

A disabled demonstration account cannot make a new sign-in. The other demonstration account can still sign in.

The described observations meet the condition. Record the result and its limit:

E2 — New sign-in check: The disabled test account could not start a new session in the N5 demonstration app; the other test account could. Existing sessions, other applications, and agent accounts were not tested.

This consumes the week’s one action. It grants no additional project progress and uses no project slot. The condition can be written as part of this Verify action. If the test facilities or authority had been missing, the group would first need to establish them through the appropriate actions.

E. Keeping earlier evidence after a change

Suppose a later project changes how the demonstration app accepts sign-ins. Keep E2 in the evidence log: it accurately describes what happened under the old arrangement. Add a new uncertainty beside the changed relationship.

The group cannot use E2 alone to claim that the new arrangement behaves identically. A later test can establish evidence for the changed scope. Nor does the change erase unrelated evidence such as E1’s training payroll results.

F. A bounded Arrival trace

Assume the group reaches the Arrival with the starter relationships still in place. Its selected scenario says the provider withdraws N4’s public outage portal. Freeze the map and assess what is already present.

Service	Trace and provisional result
Field repair coordination	N3 still supplies N1. The ordinary human phone/radio route to crews and N8 does not use N4. Operating for the described next shift, unless another established burden or fact prevents staffing it.
Outage communication	The public portal disappears. N1’s ordinary staffed phone route remains, so some reports and explanations continue. Degraded. The capacity to handle all displaced portal users is unknown; mark that limit rather than claiming an untested substitute serves everyone.
Payroll and supplier payments	N3’s output and finance staff’s ordinary route to N10 do not depend on N4. Operating under the established circumstances. This says nothing about surviving a separate loss of identity or payment services.

The existing human contact for crews continues. This scenario does not test a loss of electronic work orders, so staffing and performance under that condition remain unproven. Asha’s school-pickup obligation still matters: the group cannot quietly invent indefinite overtime to claim the extra customer calls are fully covered.

One final sentence might be:

“Mateo still answers the crews, but I do not know who will answer the customers when I leave to get my daughter.”

The surviving service, the unresolved burden, and the remaining uncertainty all belong in the ending. No new project or last-minute verification is added.

12. Using the game with an IT team

Play a fictional organization. The table’s job during the story is to make its choices and consequences understandable. After the ending, set aside 15–20 minutes to reflect on what the experience suggests.

Useful debrief questions include:

Which people and services received the most care?
What did the map reveal that an equipment list alone would have missed?
Where did implementation outrun verification?
Which AI delegation created useful capacity, and which became difficult to explain?
Which containment, recovery, simplification, or manual practice mattered at the Arrival?
Which burden persisted because its cost fell on someone whose voice was easy to overlook?
Did the initial commitment influence actual choices?

If the group wants to take something into its work, identify three follow-up experiments. Each should have an owner, an assumption to test, and an observable result that would change a real decision. The game’s fictional evidence does not substitute for evidence from that real experiment.

Follow-up experiment	Owner	Assumption	Observation that would change a decision
1
2
3

Notes for a playtest

Record a few observations after play: total time including setup and the ending; which actions were chosen; whether nontechnical players could contribute; whether obligations returned to the story; and whether the Arrival followed intelligibly from the map.

Notice which card alternatives people select and why. A preference for gentler events can be an intentional storytelling choice. Repeatedly choosing an option only because it bypasses a consequence may suggest that a pair needs revision. Evaluate the full and short calendars separately, including their different AI scheduling benefits.

13. Table reference and worksheets

One ordinary week

Draw: Choose one event option. Resolve its situation, then apply the shared instruction once. The King of Spades stops play immediately.
Progress: Preexisting RUNNING or BLOCKED projects that are now RUNNING and not paused lose one week. At zero they become READY and their recorded implementation occurs. Newly started projects never tick this week.
Act: Take exactly one action from the table below.
Record: Update findings, obligations, evidence, and questions; clear one-week pauses; rotate the steward.

Action	Result
Survey	Investigate one chosen uncertainty. Record information, without implementation or a new demonstration.
Build	Start one scoped project in an empty slot, or cancel one project as the entire action.
Verify	Test one READY project or an existing capability with available test arrangements. Pass records scoped evidence; only a declared READY target can free its slot.
Council	Give each player one affected voice, then record a decision, obligation, or acknowledged disagreement.

Project limits: Three total slots, including RUNNING, BLOCKED, and READY. Durations are 2/4/6 weeks in the full game or 1/2/3 in the short game. Suitable AI assistance reduces the initial duration once by one, minimum one; it does not reduce rework.

Shared pauses: If the event already BLOCKED a project that was RUNNING at the draw, it satisfies that card’s shared pause; do not pause a second project. Its actual block remains until its prerequisite returns.

Verify Gap: A READY project returns to 2 weeks of rework, or 1 in the short game, and stays BLOCKED if a prerequisite is unavailable. A gap on an existing capability records a finding without starting a project. A READY project may use Verify to record a preparation Gap if its test cannot yet run.

Map: PLANNED grants no capability. An existing change grants no automatic evidence. Evidence names an observation, scope, and relevant conditions. Keep earlier evidence when circumstances change and mark the new uncertainty.

People: Concerns create no score, veto, or delay. Record actual commitments and resource accommodations. Revisit an existing burden at each phase transition and the original commitment at the Arrival.

Arrival: Freeze improvements. Use the choice-or-roll method set at setup. Rotate service narrators and checkers. Trace the three services; record Operating, Degraded, Interrupted, or Unknown with a basis. Return to the original commitment, narrate the consequences, and end with one sentence per player.

Organization worksheet

Field	Our record
Organization and purpose
Essential service 1 and human outcome
Essential service 2 and human outcome
Essential service 3 and human outcome
Initial commitment
Two strengths
Two shortages
Full or short calendar
Arrival method: choose or roll

Project worksheet

Copy this for each of the three slots.

Field	Our record
Project name and scope
Fictional accountable owner
Intended change
Implementation environment
Human cost and accommodations
Prerequisites and any unresolved blocker
Written acceptance condition and test
AI assistance, authority, dependencies, and revocation, if used
Starting duration and remaining weeks
State: RUNNING, BLOCKED, or READY

Evidence and obligations

Evidence number	Observed result	Scope and relevant conditions	Remaining uncertainty or later change
E1
E2
E3

Person or team	Promise or burden	What would address it?	What happened later?

14. Small glossary

Term	Meaning in this game
Acceptance condition	The specific outcome a test must demonstrate for a claim to pass.
Agent	An AI-enabled system given a task and some ability to act. Its actual permissions determine its reach.
Boundary	A limit on communication, access, authority, or operating scope. State what the boundary limits.
Control	A safeguard intended to prevent, limit, reveal, or help recover from a problem.
Credential	A password, key, token, or other proof used to obtain access.
Dependency	Something a service or piece of work needs in order to function.
Evidence	A recorded observation supporting a claim within a defined scope and set of conditions.
Fallback	Another existing way to deliver an essential outcome when the ordinary route is unavailable.
General intelligence	The fictional broadly capable AI whose arrival changes the organization’s circumstances at the end of the year.
Identity service	The system used to establish who can sign in and what access they may receive.
Implementation	Making the intended change in its recorded environment. Its effect still needs appropriate evidence.
Model	The underlying AI component whose behavior may change when its version, instructions, or reference material changes.
Operational environment	Systems and people involved in running a physical process, such as grid control. Draw its actual relationships separately.
Prerequisite	Something that must be available or agreed before particular work can proceed.
Restore	Recover data or a service from an existing recovery source and make it usable in a stated environment.
Service account	An account used by software or automation. Its permissions may be shared by several tasks.
Update channel	The route through which software changes reach a system.

15. Revision notes and credits

What changed in version 0.2

Clarified shared authorship, speaking rights, and the authority of events and each action.
Added visible PLANNED notation and defined where READY implementation exists.
Added verification of existing capabilities, preparation Gaps, bounded evidence claims, and explicit prerequisite handling.
Preserved historical observations while allowing later conditions to change.
Made cancellation use a whole action, so it cannot release a READY slot and start replacement work at once.
Reworked the full 52-card deck into stacked alternatives with visible OR separators; retained a common delay on 5♦ and 7♠ and a common immediate ending on K♠.
Added more concrete consequences, legitimate differences of perspective, and moments of relief; revised the pilot prompts on K♣.
Made resources, obligations, phase reflections, and the initial commitment recur during play and at the ending.
Established the Arrival selection method at setup and added a repeatable service-tracing procedure.
Clarified session budgets and the short calendar’s different AI scheduling incentives.
Added the complete North Valley Electric starter, worked examples, table reference, worksheets, glossary, and clean Markdown formatting.

Design credit and further reading

Avery Alder, The Quiet Year, Buried Without Ceremony. Inspiration for shared mapmaking, community perspectives, and a year structured by card draws. This game is unofficial and does not reproduce the original event deck.
CISA, Cross-Sector Cybersecurity Performance Goals. Further reading on connected practices for understanding, protecting, detecting, responding to, and recovering from cybersecurity risks.
NIST, AI Risk Management Framework. Further reading on incorporating AI risk and trustworthiness considerations into design, development, use, and evaluation.

The countdowns, constraints, narrative incidents, and Arrival outcomes are fictional game mechanics. All organizations and people in the starter scenario are invented. Playtesting should examine the game’s clarity, pacing, and experience; its ending is not a forecast about an actual organization or the arrival of general intelligence.

Makers vs. Spectators: Five Skills for 2027

2026-10-11T00:00:00-07:00

The enterprise examples in this post are hypothetical.

Imagine being ready to sell something to a large company and discovering that the hardest part is figuring out how to begin.

Procurement sends one set of instructions. Legal sends another. Cybersecurity has a questionnaire. Accounts payable needs information that appears to overlap with something already submitted. Each document makes sense to the department that wrote it.

The supplier still cannot tell what happens next.

Inside this hypothetical Fortune 500 enterprise, everyone has done their part. Outside it, someone has to reconstruct the organization before they can do business with it.

That process has authors. Someone chose the questions, the sequence, the systems, and the handoffs. Those decisions were made under particular conditions, with particular tools and assumptions. Over time, they became familiar enough to feel permanent.

But they can be examined. They can be understood. They can be changed.

That realization is where making begins.

A spectator can recognize the problem, discuss it, attend a demonstration, and leave with an informed opinion. A maker accepts responsibility for carrying an improvement into the world.

These are ways of participating. The same person moves between them. Watching is useful. Reading, listening, and studying other people’s work are essential. The worker becomes the spectator when observation repeatedly replaces the difficult work of choosing, attempting, refining, and finishing.

As enterprises prepare for 2027, AI gives people more ways to explore an idea and make a first attempt. A process analyst can demonstrate an alternative. An operations team can examine a reporting problem. A small group can carry work that previously depended on several separate teams.

That creates a practical question: what will people do with the additional capacity?

A company can become very good at generating possibilities. It can fill its calendar with demonstrations and its planning system with promising initiatives. The work becomes consequential when someone chooses a particular result and stays with it long enough for another person to benefit.

Five skills help make that movement possible: taste, agency, focus, resourcefulness, and follow-through. Each shapes how a person sees the world. Each also shapes what that person leaves in it.

Taste: Expose Yourself to the Best of Humanity

Taste develops through exposure to the best of humanity, across cultures, disciplines, and experiences. You encounter excellent work, give it close attention, and let it change your standards. Don’t let these best practices be merely admired. Don’t let people say that won’t work where you work. All too often you hear people say, “In theory … but in practice … So we don’t do that.” These are the voices of the spectator, the critic, and the cautious.

The attitude you need requires you to see a world larger than your immediate professional surroundings, it requires a sense of humility and a willingness to be wrong and to admit you can and should learn from others. This is the first step in developing strong taste.

So, go study a well-designed building. Read carefully edited prose. Listen to music closely enough to notice its pacing and restraint. Examine a useful tool, a thoughtful service, or a piece of craftsmanship. Look at work from another place or another period. Stay with something unfamiliar long enough to understand what makes it good.

What decisions are hidden inside it? How does it guide attention? What has been removed? What does it make easier for the person encountering it?

Those questions turn admiration into material you can use.

A person early in a career can develop strong taste in a particular area through curiosity, attention, and experimentation. Years of experience become valuable when they are examined and absorbed. Simply accumulating them does not guarantee a better judgment.

Now return to our struggling supplier.

The onboarding team could begin by rearranging the existing documents. It could also look outside the company for better ways to help a person navigate unfamiliar information.

A clear museum exhibit has to introduce a complex scientific subject without demanding that every visitor arrive as an expert. Public wayfinding has to help someone choose a route while standing in an unfamiliar place they have never been to. A library has to make a large collection discoverable to someone who doesn’t quite know what they need. Thoughtful hospitality anticipates the uncertainty of someone entering a setting they do not yet understand.

These encounters suggest useful questions for the enterprise. Can a supplier recognize their situation? Can they see the sequence? Is the next action obvious? Does specialized language appear with enough explanation to be useful?

The team builds an onboarding experience around those questions. Procurement, legal, cybersecurity, and finance still have responsibilities. The supplier receives a coherent path through them.

Then someone tries it.

A new supplier understands the explanations but cannot tell which requirements apply to a service provider working at a company location. A site purchasing coordinator notices that the proposed sequence does not match how purchasing actually proceeds.

The team revises the experience.

Exposure to excellent work in other domains provides standard of what good work looks like. Contact with actual people reveals where those standards need to be applied. Taste becomes more precise through both.

For leaders, this means giving informed judgment room to become work. Someone close to a frustrating process may see distinctions that a planning meeting misses. Let that person demonstrate an alternative. Encourage references from outside the enterprise. Examine the care and clarity of the resulting experience.

An internal service deserves this attention. It shapes how people spend their working lives.

How AI Helps With Taste

The onboarding team uses AI to explore different information structures, draft explanations, and identify inconsistencies. It can compare several ways of presenting the same process and examine what each emphasizes.

The team might ask for a version organized around departments, another around supplier situations, and another around the sequence of decisions. Comparing them makes assumptions visible.

Members judge those alternatives against their understanding of excellent work and the needs of suppliers. They continue examining work beyond their professional field and observing people using the process.

AI supplies more material to consider. The people making the experience remain responsible for its judgment, coherence, and care.

Agency: Recognize That the Enterprise Is Malleable

Agency begins with a simple realization: much of the environment around you was built by people no smarter than you.

You can learn how it works. You can influence its shape. You can build things other people use.

An enterprise’s scale can make this difficult to remember. A workflow arrives with a system name, an owner, a history, and several dependencies. It feels established. But inside it are choices about what should happen, who should act, and what information should be available.

Understanding those choices and empathising with the people who wrote it, makes useful change possible.

Consider another part of the same hypothetical company. Field supervisors receive a maintenance status report every Friday. Someone assembles it manually from information that becomes available throughout the week. The supervisors then send it to leadership by the end of the day.

An operations analyst asks why do I need to do this on Friday? The answer leads back to an older process for collecting updates in order to allow leadership to review the information on Monday morning before making decisions at a standing meeting that is now on a Wednesday. The schedule for the report survived even though the consumers of the reports no longer followed the same timeline.

The analyst works with supervisors and the system owner to demonstrate a shared view that reflects the available information more promptly. An inherited reporting schedule becomes a design question.

The first version ranks maintenance work orders by age.

The supervisors explain that age alone does not identify the work needing attention. Their concern is work scheduled for the coming week that lacks a necessary part, an available crew, or a confirmed access window.

The first attempt is imperfect. It is also useful. It gives people something concrete to examine and helps them articulate knowledge they had struggled to explain in an abstract requirements discussion.

The analyst revises the view around readiness for scheduled work.

Building has clarified the plan.

This is an important consequence of agency. You do not have to understand every step before taking the first useful one. Some understanding becomes available through the attempt itself.

A team can spend a long time agreeing that its reporting is inadequate. It can gather resources, discuss possibilities, and wait for a operational improvement program to be launched. Agency appears when someone maps the process, examines the available information, and makes an alternative that can be tested.

The result may also change the maker’s ambition. A reporting improvement could reveal a problem in the handoff between planning and field teams. The next attempt begins with a better understanding of the people, systems, and decisions involved.

The environment becomes more understandable as it becomes open to informed influence.

Over time this approach can lead to a culture where informed influence is expected and valued, making the organization more adaptive and responsive to change.

How AI Helps With Agency

The analyst uses AI to turn process notes and system documentation into a draft explanation of how reporting works. That explanation helps identify assumptions to examine with supervisors and the system owner.

AI can help develop alternative report structures and a first version people can react to. As supervisors clarify what matters, the analyst uses it to revise the proposal and organize unanswered questions.

An unfamiliar technical step becomes something to investigate. The analyst can ask for an explanation, examine it against the underlying information, and bring a better question to the person with the relevant expertise.

The maker chooses the change to pursue, tests the understanding with the people involved, and takes responsibility for moving the work forward.

Focus: Say No to Good Ideas So You Can Say Yes Clearly

Focus means saying no to a hundred good ideas and clearing your mind and commitments so you can say yes to the one thing that matters most.

The difficulty is that those other ideas may be worthwhile.

A diverse world supplies possibilities. Taste makes more of them visible. Resourcefulness gives you additional ways to pursue them. Eventually, you have to decide which possibility deserves the work that comes after its exciting beginning.

Return to procurement. The leadership team has several credible proposals: AI-assisted contract summaries, supplier performance dashboards, better purchasing search, improved supplier onboarding, and automated responses to routine questions.

Each has a plausible benefit. Each has an interested sponsor. Each could consume the attention of capable people.

The team chooses to improve the path from a supplier’s initial application to readiness for a first purchase and payment. It defers the other ideas and gives that outcome a clear owner, a small team, and sustained attention and.

Now decisions have something coherent to serve.

Does a proposed feature help a supplier complete that path? Does it help an employee resolve the next step? Does it remove confusion or create another place where the process can fragment?

Focus requires working through those questions until the answer is clear enough to guide action.

It also requires creating room to think. Conflicting commitments occupy the mind long after a meeting ends. An initiative that has supposedly been deferred can remain an obligation if nobody clearly tells its sponsor what was decided. A team can have one official priority and several unofficial ones.

Clarity in the result begins with clarity in the commitments behind it.

Imagine two teams with similar tools. One spends a month producing demonstrations across several procurement topics. Each demonstration attracts interest, and each new request creates another beginning.

The other stays with supplier onboarding. It works through departmental handoffs, brings suppliers through the revised process, and learns what still prevents completion.

Both teams are producing artifacts. The second team’s attention accumulates into a deeper understanding of one problem and a result that reaches people.

Curiosity still matters. Wide interests help makers discover worthwhile problems and useful approaches. The discipline is to keep curiosity broad while being selective about commitments.

A prototype should answer a question, improve a decision, or advance the chosen work. When it repeatedly creates another possibility, ask whether it is helping the commitment develop.

Leaders have work to do here. Make deferred ideas visible. Resolve competing requests. Support the team after the novelty fades. Review progress through the intended outcome and the decisions still required to achieve it.

A meaningful yes includes protecting the work from other attractive demands.

How AI Helps With Focus

Leadership gives AI the proposed initiatives, current commitments, and criteria that matter to the enterprise. AI helps organize proposals around intended outcomes, identify overlapping work, and summarize competing demands.

That comparison gives the team material for examining tradeoffs. It can also reveal questions that need investigation before a decision is made.

Once supplier onboarding is chosen, AI helps refine a clear scope statement and identify activities that do not advance it. When new requests appear, the team can compare them with the agreed outcome and decide whether to include or defer them.

AI helps make the choices understandable. Leadership makes those choices, communicates what will be left for later, and protects the attention required to finish.

Resourcefulness: Bring a Wide World to a Difficult Problem

Resourcefulness combines diverse experience, unconventional approaches, and the strengths of small, elite teams to find a way forward.

You cannot know in advance how every experience will matter. The connections become clear as you look back.

This gives curiosity a role beyond an immediate business case. An encounter with a craft, a culture, or an unfamiliar discipline may have little obvious relevance when it happens. Years later, a problem gives it a use.

The supplier onboarding team brings together five people: a procurement specialist, a service designer, an integration engineer, an accounts payable analyst, and a site purchasing coordinator. It draws on legal, cybersecurity, and other process owners when their expertise or decisions are needed.

Each person understands something the others need.

The designer previously studied museum exhibits knows how difficult it can be to introduce unfamiliar information. The site coordinator once managed event logistics and recognizes how an unclear handoff can disrupt an otherwise thoughtful sequence. The engineer has worked on incident response and knows how unresolved ownership can hide inside a queue.

Those experiences were not acquired as preparation for supplier onboarding. Their relevance emerges through the work.

The group also shares demanding standards. Members can challenge assumptions, revise their own thinking, and contribute strong craft. “Elite” describes the quality of their judgment, learning, and commitment.

A small team becomes capable through the combination of its members and access to the people whose participation the problem requires.

At first, this team assumes that the onboarding experience will require a substantial new platform. As it examines the process, it finds useful capabilities already present in existing systems. It also finds redundant requests, confusing explanations, and handoffs that can be redesigned.

The solution becomes a combination of clearer information, revised responsibilities, and better use of existing capabilities.

The outcome the team is working on stays the same. The path they take to reach it changes.

Resourcefulness depends on being able to make that adjustment. The maker cares enough about the purpose to reconsider how it should be achieved.

Learning continues near the point of need. The procurement specialist learns enough about information design to evaluate the experience. The engineer learns why particular payment requirements matter. The designer learns how purchasing differs between business units.

Each explanation has a concrete use. It serves a problem the team cares about.

Finishing the work adds another set of experiences. Looking back, members can see which earlier ideas mattered, where a colleague’s perspective changed the result, and what they are better prepared to attempt next. This reflection can only be done after the product is shippped. It is an essential part of developing the team.

For leaders, the useful question is how people behave when they reach unfamiliar territory. Do they find a useful question? Reframe the problem? Seek help? Return with another informed attempt?

Give small teams access to the expertise and decisions their work requires. Let them question the expected route while keeping the purpose clear.

How AI Helps With Resourcefulness

The team uses AI to explain specialist terms and compare how departments describe the same process. This can make it easier for members to understand one another’s concerns.

They can explore possible connections between information design, logistics, incident response, and supplier onboarding. AI helps generate candidate approaches using the capabilities and constraints the team has identified.

When an unfamiliar technical or procedural problem appears, AI helps explain it and prepare useful questions for the relevant expert.

It can also support reflection. After a delivery milestone, the team uses AI to organize decisions and feedback, then examines how earlier experiences contributed to the result.

The people supply lived experience, judge the proposed connections, and choose the route that serves the purpose.

Follow-Through: Do the Work and Ship It

Great ideas are cheap. Follow-through is the passion and discipline to do the grueling work that turns an idea into something excellent, finished, and usable.

An idea can be compelling before any of its difficult details have been resolved. A demonstration can make the result look close.

Then the details arrive.

Information does not reach the right system. A person does not know who owns an exception. A sequence works for one business unit and becomes confusing in another. An explanation has to be rewritten. Someone has to prepare the people who will operate the service.

Given all these challenges, it will be tempting to move the goal posts and to cut corners. With each adjustment, the team risks undermining the very experience it is trying to create. Discipline and commitment are required to maintain the integrity of the work through these difficulties.

Passion matters at this point because it gives the work a reason to continue through this period. Care has to survive the loss of novelty and the temptation to settle for something short of a bullseye.

The supplier onboarding team demonstrates a clear, attractive experience. During the pilot, suppliers complete the intake successfully. But some information does not reach the purchasing system correctly. Site buyers remain uncertain about who handles exceptions. Accounts payable discovers that a completed intake does not always provide everything needed to prepare for the first payment.

The team stays involved. Just because the project is “over” in the sense of the initial rollout does not mean the work is complete.

It corrects the information handoffs, clarifies responsibility, revises instructions around the purchasing sequence, prepares employees to use the process, and establishes continuing ownership.

These tasks determine whether the experience holds together. They are part of making the idea real.

This is what Steve Jobs meant when he said: “Real artists ship.”

Shipping requires bringing work to a condition where its intended recipients can use or experience it. It requires decisions about what belongs, resolution of important problems, and completion of the path into use.

There are layers to that completion. The work has to exist. Its details have to meet the intended standard. It has to reach people in their actual circumstances. It has to fulfill its purpose. Someone has to carry responsibility for its future. The user has to be able to engage with it successfully to achieve their intended outcome.

A maker follows the work through those layers. A spectator does their small bit, and throws the work over the fence.

For the supplier, success means understanding what is needed and being able to complete the process. For the site buyer, it means knowing the status, the next action, and who can resolve a problem. For procurement and finance, it means receiving the information necessary to carry out their responsibilities.

The whole job exists across those relationships.

This also changes how colleagues come to trust someone. They learn whether that person remains involved after a promising beginning. Process owners see whether difficult details receive care. Recipients experience whether the work delivers what was promised.

A reputation develops through the quality and reliability of completed contributions.

How AI Helps With Follow-Through

The team uses AI to organize pilot feedback into problems involving explanations, information handoffs, responsibilities, and exceptions. It helps prepare test scenarios that follow different suppliers through the complete process.

AI can assist with revised instructions, employee training materials, and guidance for the people who will operate the service. The team compares those materials with the implemented process and looks for gaps or inconsistencies.

After release, AI helps summarize recurring questions and reported problems. That gives the team another way to identify what deserves attention.

The maker validates the work, coordinates its release, observes its use, and remains responsible for the promised result. AI helps carry the tasks. Passion and commitment keep the people engaged with the purpose.

What Will You Make?

These skills operate together. Feedback changes taste. An attempt clarifies the next action. A new possibility demands another choice. An unfamiliar problem brings someone’s earlier experience into use.

They also reveal why a promising effort can stall. Perceptive judgment can remain criticism when nobody acts. Initiative can create endless beginnings when attention is scattered. Concentration can become frustration when a team cannot find another route. Clever solutions can remain demonstrations when nobody carries them into everyday work.

The practical challenge for 2027 is to bring a wide world into a clear commitment.

That requires something from leaders as well as individuals. Give people close to problems room to demonstrate improvements. Encourage them to encounter excellent work beyond the enterprise. Assemble small groups with complementary strengths. Help them choose an outcome and reach the people whose decisions it requires.

Then stay interested during the difficult period between an impressive first version and useful adoption.

There is probably a project your organization has learned to tolerate leaving unfinished. There is a team that engaged with a step of the solution, passed it along to the next team, and then moved on. Perhaps there is a report that arrives too late, a service that makes employees understand the organization before they can get help, or a process that asks suppliers to carry its complexity.

Start there. This is the more valuable work that AI will enable us to accomplish.

Examine what excellent work could teach you. Look at the inherited decisions being treated as permanent. Decide which other good ideas must wait. Bring the right experiences and people to the problem. Ask whether the purpose matters enough to sustain the work after the excitement fades.

Then make an attempt that reveals something. Follow it into use. Let the finished contribution change your understanding of what you can attempt next.

Return to the supplier at the beginning of this post. The difference becomes visible when that person can open the instructions, recognize their situation, and know what to do next. The departments can carry out their responsibilities. The purchasing process can proceed.

Someone has given a piece of the enterprise enough thought and care to make it coherent.

Working toward that result also gives you opportunities to grow. You encounter a distinction you could not previously see. You question an arrangement you once accepted. You learn what deserves attention, whose experience can help, and what finishing actually requires.

Useful practice can also happen outside the project. Comparing interpretations of a piece of music can sharpen judgment. Learning an unfamiliar craft can teach you how to proceed when the instructions stop being sufficient. Returning to something another person has used can reveal what you overlooked when you first called it finished.

Those experiences become part of what you bring to the next attempt. Their connections may become clear only when you look back.

So choose a contribution worth making, and give your development a deliberate place in the year. Small exercises, encounters with different kinds of excellence, and repeated attention to the same piece of work can strengthen the capacity to finish it well.

What will you make in 2027? And how will you use the year to become more capable of making it?

How Will You Get Better: A Year of Practice

One way to strengthen these skills is to spend time in situations where your usual expertise provides little help.

Taste can grow through comparing performances of the same piece of music. Agency can grow through examining a household routine. Resourcefulness can grow while making something with your hands and discovering that the original plan does not work.

These activities bring different experiences into your judgment. Their usefulness may become clear much later.

Set aside a few focused hours each month. Keep a short record of what you tried, what surprised you, and which decision changed. Several months below return to the same contribution, giving the year a continuous thread of making, observing, and improving.

January: Taste Through Comparison

Choose one work and examine three interpretations: performances of the same music, translations of a poem, or productions of a scene. Include an interpretation from a tradition unfamiliar to you.

Describe the choices that make each version effective. Pay attention to pacing, emphasis, restraint, and what has been left out. Finish with a short note about one principle worth carrying into your own work.

February: Agency Through an Inherited Rule

Choose a recurring meeting, household routine, or shared process that persists because it is familiar. Find out who established it and what problem it originally solved.

With the people involved, test one small change for two weeks. Record what improved and what remained necessary. The exercise makes a human decision visible inside something that previously felt fixed.

March: Resourcefulness Through Unfamiliar Craft

Make or repair something modest using materials already available. Bind a small notebook, mend a garment, or try a craft you have never practiced.

Choose something that requires learning as you go. When the plan breaks down, investigate the next step and adapt. Record one moment when a constraint led to a better approach. That experience may become useful in an entirely different setting.

April: Focus Through Subtraction

Choose one existing commitment that deserves sustained attention. Then identify three worthwhile optional activities competing with it.

Pause or defer those activities. Write down why each is attractive and why the chosen commitment matters more right now. Protect a regular period of uninterrupted attention.

This exercises the difficult part of focus: declining something good.

May: Follow-Through for One Person

Choose one person with a small recurring problem. Make something useful for them: clearer instructions for a volunteer, a checklist for a colleague, or a simple guide for a community group.

Watch the person use it. Resolve the confusion that appears and put the revised version back into their hands. Finish the month with something another person has successfully used.

June: Taste Through Someone Else’s Eyes

Ask someone whose craft you know little about to explain how they judge quality. A musician, gardener, editor, cook, mechanic, or illustrator could provide a different way of seeing.

Ask them to compare two examples and explain the details they notice. Capture their criteria, then revisit January’s exercise. Look for distinctions you can now recognize that you previously missed.

July: Agency Through a Small Redesign

Choose part of your immediate environment that you can change: a workspace, a form you own, or the sequence of a routine.

Before changing it, describe what the current arrangement asks people to do. Build an alternative and try it for a week. Observe whether it makes the intended action easier.

Agency grows through experiencing a connection between a deliberate change and someone’s ability to act.

August: Resourcefulness Through a Small Team

Return to May’s contribution. Invite two capable people with different strengths to help improve it.

One might understand the recipient’s needs deeply. Another might bring strong writing, design, or technical craft. Give them room to reshape the work and challenge your assumptions.

Record which decisions became better because of their contributions. Practice assembling a small team whose combined judgment exceeds what you could bring alone.

September: Focus Through Editing

Write the purpose of that contribution in one sentence. Examine every feature, paragraph, and step against it.

Remove something you like that does not adequately serve the purpose. Try the simpler version with its intended user and observe what happens. Keep the change when it improves clarity or usefulness.

This develops focus at the level of the finished work. Selection becomes visible in what remains.

October: Follow-Through by Returning

Go back to the person using the contribution. Ask what they still work around, explain to others, or complete outside it.

Observe the work in its actual setting. Choose one unresolved friction and carry the improvement through to use.

Returning after several months exercises a different kind of commitment. The work has become ordinary, and the recipient’s experience still deserves attention.

November: Resourcefulness Through Hindsight

Read the notes collected during the year. Look for earlier experiences that unexpectedly influenced later decisions.

Perhaps March’s failed attempt helped you approach an unfamiliar problem. Perhaps June’s conversation changed an editing decision in September. Trace the connections through what actually happened.

AI can help organize the notes and suggest patterns to examine. Judge those patterns against your record. The exercise is to recognize connections that emerged through living and making.

December: Put the Five Skills Together

Choose one existing, unfinished commitment small enough to complete this month.

Bring your stronger standards to it. Examine an inherited assumption. Give it a clear purpose and enough attention. Seek the experience or collaborator needed for its difficult parts. Then finish it and put it into use.

Record what the intended person can now do, understand, or experience better. Let that result inform what you choose to attempt in 2028.

The useful record of this year will include encounters, changed decisions, declined possibilities, unexpected connections, and finished contributions.

By December, you should have something more concrete than a claim that your skills improved: a piece of work another person can use, and a clearer understanding of how you became capable of making it.