The Playbook

Most failed AI features die here, before a line of code, because the team never asked whether the problem needs a model at all. A deterministic rule that is right every time beats a model that is right most of the time, and the honest first question is which one your user actually needs. Framing means writing down who the user is, what a correct outcome looks like for them, and what a wrong one costs them, in their terms rather than yours.

The discipline that makes this real is defining good before you build. Teams that skip it end up arguing about vibes in a review meeting, because everyone evaluated the demo against a private, unstated bar. Write the bar down first: what the product must do, what counts as failure, and how you would know the difference. Every later step in this manual checks work against that page.

Taught in the Builder’s Stack:Find your build·Define what good means

The industry converged on this practice from both ends: large labs publish model specs that state intended behavior in plain language, and product teams learned that prompt-tinkering without a contract produces regressions nobody can name. The behavior contract is the product decision; the system prompt is merely its current implementation. Keeping those two separate is what lets you change models, or prompts, without losing what your product promised.

Write the contract as testable statements: what the product always does, what it never does, and what it does when unsure. Then derive the system prompt from it, and treat every hand-written rule in the prompt as a claim the contract must already contain. When a stakeholder asks for a behavior change, the edit lands in the contract first, which is also what makes the escalation rules explicit rather than tribal.

Taught in the Builder’s Stack:Prompt like an engineer·Output your code can use

Model choice is a product decision wearing an engineering costume. The real questions are the user's: how good does this have to be, how fast does it have to feel, what may it cost per use, and what happens when it is wrong. Benchmarks answer almost none of that, because public leaderboards measure general capability while your product needs specific behavior on your inputs. The only trustworthy comparison is your own eval set run against two or three candidates.

Grounding is the other half of the decision: what the model must know that it was not trained on, and how it gets that knowledge. Retrieval, tools, and context each carry costs and failure patterns, and the choice among them changes your architecture more than the model choice does. Write the rationale down, because the model market moves fast, and a decision with a recorded why can be revisited in an afternoon when the market does.

Taught in the Builder’s Stack:Choose a model·Give it the right context

A working prototype is the cheapest eval you will ever run. Specifications for probabilistic systems are reliably wrong in their edge cases, because nobody can predict where a model breaks by reading about it; you find out by typing real inputs into a real loop. Building one yourself, in a day, is now realistic for a PM with modern tools, and doing it yourself matters: the surprises are the point, and secondhand surprises do not teach.

Keep the prototype honest about what it is. It exists to find the cases your spec missed, not to become the product, so wire the real model with real data pathways and skip everything else. Every strange output you hit becomes a line in the edge-case list, and that list seeds the eval suite in Ship, which is how a day of prototyping quietly becomes your first test corpus.

Taught in the Builder’s Stack:Make your first call·Wire in the model

Guardrails exist because the model is not the control layer, whatever the demo suggested. Injection attacks, hostile inputs, and plain bad luck all reach production, and the security work of the last few years, catalogued publicly in the OWASP list for language-model applications, keeps landing on the same conclusion: filter what goes in, check what comes out, and never let the model authorize its own actions.

Build them as system behavior, not prompt requests. An input filter decides what the model never sees; an output check decides what the user never sees; an escalation path decides which situations get handed to a person by design. The rollout checklist is the operational guardrail: staged exposure, a kill switch someone can actually reach, and a named owner for the first bad week.

Taught in the Builder’s Stack:Guardrails·Defense in layers

Evals are the AI era's test suite, and the teams that ship reliably treat them exactly that way: written early, run on every change, and grown from every failure. The reason is structural. You cannot catch quiet failures by eye, and a model change that improves the average can still break the exact cases your users depend on. A graded set of real cases with pass and fail thresholds is the only instrument that notices.

Start smaller than feels serious: twenty cases drawn from the prototype's edge-case list beat two hundred invented ones. Grade what matters, not what is easy to grade, and be careful with model-graded evals, which are useful at scale and quietly biased in ways human spot-checks must keep calibrating. The release gate is the point of it all: a change that fails the suite does not ship, which turns quality from a debate into a mechanism.

Taught in the Builder’s Stack:Build your eval·Block bad changes

The human-AI interaction research of the last decade, from the widely cited design guidelines out of Microsoft to Google's People + AI work, keeps returning to one principle: systems earn trust by being honest about uncertainty, not by hiding it. Users calibrate on the failures they can see. A product that shows sources, flags low confidence, and makes undo effortless teaches its users when to lean in and when to check.

Design the uncertain moments on purpose. Preview before commit for anything consequential, undo for anything committed, sources for anything factual, and a visible way to say the answer was wrong. These patterns cost little and compound: every recovered mistake is a moment the product proved it was built by someone who knew mistakes would happen.

Taught in the Builder’s Stack:Help users catch errors·Show what it can do

Every response your product gives has five properties in tension: relevant, consistent, appropriate, affordable, and fast. You cannot maximize all five, and pretending otherwise just means the trade-off gets made by accident, usually as a latency spike or an invoice. A budget is the written form of the trade-off: what a response may cost, how long it may take, and which of the five you deliberately hold looser.

The levers are now standard practice across the industry: route easy requests to smaller models and hard ones to larger, cache what repeats, and cap the loops an agentic flow may take before it must conclude or escalate. Set the budget per feature rather than per product, because a drafting flow and a checkout flow do not tolerate the same latency, and revisit it whenever the model or the traffic changes.

Taught in the Builder’s Stack:Cost per task·The Inference Budget

Traditional monitoring watches whether the system is up; AI products need monitoring that watches whether the system is right. Those are different instruments. A dashboard of uptime and latency will read green while answer quality degrades, so the quality dashboard tracks what users actually experience: refusal rates, correction rates, escalation rates, and the metrics your definition of good named back in Shape.

The practice that separates teams who catch problems from teams who read about them in reviews is session review: a standing habit of reading real transcripts, on a cadence, by people empowered to change the product. Sampling a few dozen sessions a week finds the failure patterns no metric was designed to count, and it keeps the team's instinct for the product's real behavior current.

Taught in the Builder’s Stack:Signals after you ship·Monitoring

Your product changes when you change nothing. Provider models are updated and retired, user behavior shifts as people learn the product, and the world the model describes moves on its own. The machine-learning operations literature has warned about silent degradation for a decade, and language-model products inherit the problem with a twist: the model under you can change without your consent or your knowledge.

The defenses are alerts and gates. Drift alerts watch your quality metrics for slow decay rather than sharp failure, which is the pattern human attention misses. The regression gate is the standing rule that no model change, yours or the provider's, reaches users before the eval suite has run against it. Together they turn model updates from a recurring emergency into a scheduled, testable event.

Taught in the Builder’s Stack:Block bad changes·When metrics get gamed

This step is the difference between operating a cycle and running a launch. Everything Track catches, a failed session, a drift alert, a user correction, is raw material: each one becomes a new eval case, and the pattern behind several of them becomes a proposed change to the behavior contract. Teams that skip this step fix incidents one at a time forever, because the learning never lands anywhere durable.

Keep the mechanics light. A weekly pass over the failures, a ranked list of the contract changes they suggest, and a decision about which one enters the next turn through Shape. The ranking is the product judgment: not every failure deserves a fix, and the discipline of choosing makes the cycle converge instead of thrash.

Taught in the Builder’s Stack:Signals after you ship·Build your test cases

A retrieval system is only as good as what it is allowed to read, and retrieval is where most enterprise AI products quietly rot. Stale documents produce confident answers about a world that no longer exists; conflicting documents produce answers that depend on which copy the search happened to find. Retrieval-augmented generation was designed to ground models in your knowledge, which means your knowledge is now part of your product.

Governance is three written rules. A curation policy says what enters the corpus and who approves it. A refresh cadence says how each source stays current and when it expires. Conflict rules say which source wins when two disagree, and that one document is retired when it is superseded rather than left to compete with its replacement.

Taught in the Builder’s Stack:Own what it reads·Keep the facts fresh·Give it the right context

The blunt version of this duty: your AI product must not tell people things they are not allowed to know. Retrieval makes this sharp, because a model with access to every document will answer any employee's question with any executive's data unless permissions travel with the content. Access rules are behavior rules, and they belong in the same contract as everything else the product may never do.

The safe-refusal pattern is the user-facing half. When the product declines, it declines legibly: it says that an answer exists but is not available to this user, rather than pretending ignorance or, worse, summarizing what it was supposed to withhold. Test refusals in the eval suite like any other behavior, because a leaky no is a breach with good manners.

Taught in the Builder’s Stack:Decide who sees what·The Security Posture·The Agent Access Policy

Once a product acts on its own, plans, calls tools, takes steps, every weakness you shipped compounds with each step it takes. A one-in-a-hundred error rate per step becomes near certainty across a long chain, which is why the practical guidance from teams building agents converges on restraint: the simplest loop that works, clear stopping conditions, and human checkpoints where actions become hard to reverse.

Supervision design decides which actions an agent may take alone, which need a person's confirmation, and which are forbidden outright. Iteration caps bound how long it may try before it must conclude or escalate. The reliability budget states the error rate you can actually live with per flow, which is the number that tells you whether an agent belongs in that flow at all.

Taught in the Builder’s Stack:Write the Agent Charter·Keep a human in charge

A cycle only you can run is not yet a practice. The first person to internalize this manual becomes a bottleneck by succeeding, and the way out is deliberate: hire and grow people against the skills the cycle actually needs, judgment about behavior, comfort with evidence, willingness to read sessions, rather than against a job title that predates the work.

The hiring rubric writes those skills down so interviews test for them. The org change plan is the honest map of how the work moves from one owner to a rotation: who runs evals, who owns the contract, who reads sessions this week. Ownership that lives in one head is a risk; ownership that lives in a document with names on it is an operation.

Taught in the Builder’s Stack:Build the team