Incident response: when your product says the wrong thing in public

Everything missing from that channel could have been decided in advance, calmly, on a single page. This chapter writes the page: what counts as severe, what the first hour holds, who speaks, and how the incident ends.

Why a behavior incident is not an outage

When a deploy takes your service down, the script is familiar: revert the diff, watch the graph recover, post the all-clear. A behavior incident denies you all three moves. There is usually no single diff to revert: the reply telling your customer to dispute the charge came from a prompt, a model version, retrieved help-center pages, and one user's phrasing, or from a provider-side model update you never shipped. The failure recurs probabilistically: you cannot make the assistant repeat the bad refund answer on demand, yet it will fire again on some fraction of similar questions. And the damage lands on trust instead of uptime, because an error page reads as downtime while a confident wrong sentence in your product's voice reads as your product's opinion, and the twelve-hundred-repost thread will still be there after the rollback.

An AI incident is judged by the response, not the event: the failure was probabilistic, the handling is not.

The loop: detect, contain, communicate, learn

On the bad day, four jobs compete for the same hour: finding out how bad it is, stopping it, telling people, and making sure it stays fixed. If you improvise, they block each other; run them in order and each one hands the next what it needs.

• Detect with your own instruments, because users rarely report. Most people who get a bad answer just close the tab. The instruments from Production signals: evals after the ship notice anyway: a thumbs-down cluster on one topic, a canary release alert, a spike in mentions. Detection ends when someone states a severity level from the ladder below.
• Contain with moves you built earlier, never moves you invent live. The release gate from Change control: ship prompt and model changes like releases is your containment kit: roll back to the last gated release, flip the kill switch if the whole product is implicated, or turn off the affected topic and route those users to a human.
• Communicate fast, factually, and under one name. Inside, one thread and one owner, the accountable name from Ownership: one name on every model behavior, posting what is known and what happens next. Outside, a plain statement of what the product got wrong and what changed, moving only as fast as your verified facts.
• Learn by turning the captured transcripts into eval cases.

The first page of the runbook

Severity for behavior incidents measures exposure: who got the wrong answer, about what, and how far it can spread. Three levels are enough. This page is written for the support assistant in the opening scene; rewrite the middle column in your product's terms and put your names in the third.

Every level runs the same moves at different urgency: freeze, contain, capture, communicate, learn. The ladder keeps them proportionate, so a snarky tone does not trip a kill switch and a viral refund lie does not sit in a ticket queue.

The reasonable objection is size: you are six people, and runbooks sound like equipment for companies with an on-call rotation. But the table above is the entire document, and the bad-day choice is one page versus improvising in a thread while the screenshot travels.

How the response ended Cruise's robotaxi business

One night in October 2023, a woman crossing a San Francisco street was struck by a hit-and-run driver and thrown into the path of a driverless robotaxi operated by Cruise, General Motors' autonomous vehicle company. The robotaxi braked but could not avoid her, and after stopping with her beneath it, the vehicle attempted a pullover maneuver and dragged her roughly twenty feet. She survived with serious injuries. The collision began with a human driver beyond any company's control.

The response was not. Cruise's first report to NHTSA, the federal road-safety regulator, filed the next day, did not mention the dragging, and when company representatives showed officials video, the footage ended at the first stop. The consequences arrived in order:

• Ten days after the crash: a second required filing omitted the dragging again.
• Three weeks after: the California DMV suspended Cruise's driverless permits, writing that footage of the pullover maneuver "was not shown."
• Seven weeks after: the CEO resigned.
• In January 2024: the review Cruise's board commissioned from the law firm Quinn Emanuel found no intentional deception, but a failure of leadership and an "us versus them" mentality toward regulators; more than a hundred employees knew about the dragging before officials were briefed.
• In September 2024: NHTSA fined Cruise $1.5 million for the incomplete reports, and that November the company admitted in a federal settlement that it had filed a false report.
• Fourteen months after the crash: GM stopped funding the robotaxi business it had put more than $10 billion into.

Run the loop over that record: detection worked, the fleet was contained, the postmortem was thorough. The response failed at communicate, where the worst fact was the one left out.

An incident is not over until it becomes eval cases

The loop can also run clean in public: when a 2025 ChatGPT update flattered and agreed with users well past honesty, OpenAI rolled it back within days and published postmortems, an episode Change control: ship prompt and model changes like releases dissects in full. The step that belongs here is the last: the postmortems committed to sycophancy evaluations in the deployment process, converting a public failure into a permanent test.

A code bug closes when the fix deploys, but a behavior incident has no single fix, so it closes when the transcripts captured in the first hour become cases in the suite behind The regression gate: no change ships blind and the current release passes them.

An incident is not closed until it exists as eval cases that every future release must pass.

Try it now

This drill takes about fifteen minutes and produces your runbook's first page.

Write the severity ladder in your product's words. Copy the three-level table and rewrite the middle column: what counts as harm or public spread for you, which owned topics make a wrong answer an S2, what you are willing to call drift.

Copy the containment move from your release gate. Open the release gate you wrote in Change control: ship prompt and model changes like releases and paste its rollback step into the contain line of every level, word for word. If there is nothing to paste, you found a second gap: write the rollback steps down now.

Name the comms owner. One person in the table, plus a backup. Their first-hour job is a factual internal post and a draft external statement, not perfection.

Put the page where the team will look mid-incident. Pin it in the incident channel and link it from the on-call notes. It becomes the incident-response section of the charter you assemble in Write your Operations Charter and run the operating review.

Chapter Summary

• A behavior incident is not an outage: there is no single diff to revert, the failure recurs probabilistically, and the damage is trust rather than uptime.
• You are judged by the response, not the event, because the event was probabilistic and the handling is deliberate.
• Run the loop in order: detect, contain, communicate, learn.
• Users rarely report bad answers, so detection depends on your production signals and ends with a severity call.
• Containment is prepared in advance: rollback from your release gate, the kill switch, or turning off the affected topic.
• Communication is fast, factual, and carried by one named owner, moving only as fast as verified facts.
• Cruise's crash was severe, but the withheld footage and incomplete reports are what suspended its permits, forced out its CEO, and drew the federal fine; the business shut down about fourteen months after the crash.
• The runbook is one page, a severity ladder plus a first-hour checklist per level; on the bad day you follow it or you improvise.
• An incident closes when its transcripts become eval cases that every future release must pass.
• The loop runs on people with the right skills, which is who Build the team: hiring for AI and raising the org's bar teaches you to hire.

Sources

• California DMV (2023). Suspension of Cruise's autonomous vehicle deployment and driverless testing permits, October 24, 2023; the "was not shown" language describing the withheld pullover footage is from this action.
• Quinn Emanuel Urquhart & Sullivan (2024). Report to the Boards of Directors of Cruise LLC, GM Cruise Holdings LLC, and General Motors Holdings LLC regarding the October 2, 2023 accident in San Francisco. January 2024.
• NHTSA (2024). Consent order with Cruise: $1.5 million penalty for incomplete crash reports under the Standing General Order. September 2024.
• U.S. Attorney's Office, Northern District of California (2024). Cruise admits to submitting a false report and agrees to pay $500,000. November 2024.
• CNBC (2023). Cruise CEO and founder Kyle Vogt resigns. November 2023.
• General Motors (2024). GM to refocus autonomous driving development on personal vehicles. December 10, 2024.
• OpenAI (2025). "Sycophancy in GPT-4o: What happened and what we're doing about it" and "Expanding on what we missed with sycophancy." April and May 2025. (All incident facts last verified July 2026.)