Write your Operations Charter and run the operating review

This part made those decisions one at a time, and each drill left its page wherever the work happened: the owner map in the repo, the release gate in the pipeline docs, the curation policy beside the connector config, the incident runbook in the on-call folder. None of it sits on a page anyone in that meeting could open. This chapter moves the whole layer onto one signed page, then sets the meeting that keeps the page true.

One page that states how the product is run

We keep a fillable Operations Charter, written per product, and its sections each compress one chapter of this part into fields you can complete in about an hour.

The Operations Charter is the operating layer on one signed page: who owns each behavior, how changes ship, what the product may read and show, what happens on the bad day, who staffs it, and the date one accountable person reviews the whole.

The rest of this chapter walks the page section by section.

The owners and the gate

The owner map. The first section is the table from Ownership: one name on every model behavior: one row for each thing that moves behavior (the spec, the prompts, the eval bar, the corpus, the access rules, incidents), one accountable person per row, and the one-line sign-off rule with its evidence. Owners are people rather than teams, no row carries two names, and a row whose honest entry is "no one" stays as a blank with a date, so the gap stays visible until someone owns it.

The release gate. The second section prints the gate from Change control: ship prompt and model changes like releases, built there after dissecting the GPT-4o sycophancy rollback: the eval suite at its bar, the row owner's sign-off by name, a written canary plan, a tested rollback, and a comms note when users will notice. The charter adds the line the opening scene was missing: where release records are filed, so "who approved this and what did the gate show" is a lookup.

What it reads and who may see it

The knowledge section. The third section holds the curation policy from Govern the knowledge: what your product is allowed to read, the page Google relearned in public when AI Overviews served satire as advice: the eligible-source table (owner, refresh cadence, conflict rank per class), the one-line conflict and removal rules, and the not-eligible rows that keep the connector defaults out.

The access section. The fourth section holds the rules from Govern access and safety: decide who may see what, the lines the Microsoft 365 Copilot rollouts taught enterprises to write: who-may-see-what, one line per requester tier; the refusal wording, verbatim, that reads the same whether the hidden thing exists or not; and where the audit trail lives, recording who asked, what was read, and what was shown.

The bad day and the team

The incident runbook. The fifth section compresses Incident response: when your product says the wrong thing in public to what the person being paged needs: the severity ladder, the first-hour steps, the comms owner by name, and the rule that closes the loop, every incident leaving eval cases behind in the suite the release gate runs.

The team section. The sixth section carries the plan from Build the team: hiring for AI and raising the org's bar: coverage (every charter row has a person, every person has a stand-in), the rubric focus for the next hire, and the training cadence that raises the rest of the org.

The operating review keeps the page true

Everything above decays: models get swapped, corpora widen, and a charter written at launch describes a different product two quarters later. So the last section is the operating review: a standing session, on a date, where the owners check the page's claims against the product's evidence. The pattern predates AI products: Google's SRE book runs services on a weekly production meeting of thirty to sixty minutes, with a chair and a standing agenda. The charter's agenda has four items:

• The production dashboards. Each behavior row's quality signals, read by the row's owner, so no instrument sits unread between reviews.
• The gate log. Every release since the last review: what changed, who signed, what the gate showed; a production change missing from the log is a side door to close.
• Incidents and their eval cases. Each incident since the last review, with the cases it left in the suite. An incident with no cases is unfinished work.
• Spend against the budget. The meter against the budget you signed in Write your Inference Budget and ship a feature that pays for itself, because a product can behave well while tripling its cost.

Monthly fits most products; go weekly while a launch or a migration is moving. The demand is written into standards too: ISO/IEC 42001, published in December 2023 as the first certifiable management standard for AI (Anthropic announced accredited certification in January 2025), requires top management to review the AI management system at planned intervals; the operating review is the same requirement at the scale of one product.

An unsigned charter is a wish, and even a signed one drifts: the operating review is the cadence that keeps the page true.

The objection: these pages already exist

The fair pushback is that every section above summarizes a document your team already keeps, each living where it is enforced, and that the second copy is the one that rots: within a quarter the charter says "monthly review" over a calendar that says cancelled. It is right about copies and wrong about what the charter holds: each section is a decision, a name, and a pointer to the living document and its enforcing system, and the drift it fears is exactly what the review agenda checks on a schedule instead of leaving an incident to find. The opening scene was not missing documents, only the one page that says where they live and whose name answers for the whole.

One person signs it

The last line is a signature: the one person accountable for how the product behaves in production, the way the Inference Budget and the Agent Access Policy each carry one. This part opened with a product whose cost had the chief executive's full attention while quality had no owner of comparable standing, in Why AI products fail on the org chart, not in the model, and the signature is the fix at your scale: whatever the sections delegate, one name answers for the whole page.

Try it now

The drill is the part's capstone: assemble and sign the whole page, in no more than an hour.

Gather the leave-behinds. Print the fillable charter and collect what this part's drills produced: the ownership inventory, the owner map, the release-gate page, the curation policy, the access rules with their refusal wording, the incident runbook, and the team plan. Most fields are already drafted in those pages.

Fill it in one pass. Work the sections in the order this chapter walked them, holding the one-hour cap, and where a leave-behind gives the answer, write it as the decision with its owner and enforcing system.

Mark the honest blanks. Where a section has no decision yet, write the open question with a person's name and a date beside it. A blank with an owner is homework; a blank without one is the next incident. Scale it down: where a field depends on instrumentation you do not have yet (a gate log, a spend meter, a quality dashboard), enter an order-of-magnitude estimate, mark it open, and make wiring the real number that owner's first task.

Sign it, set the date, book the review. One name at the bottom, a review date at most a quarter out, and the first operating review on the calendar, agenda pasted into the invite, every owner on the map invited.

Chapter Summary

• This part's decisions were made one at a time and filed where they were made; the Operations Charter moves them onto one signed page.
• The charter holds decisions, names, and pointers rather than copies: the one-line rule, the accountable owner, and where the enforcing system lives.
• The owner map puts one person on every row that moves behavior, and honest blanks stay visible with owners and dates.
• The release gate section names where release records live, so "who approved this and what did the gate show" is a lookup.
• The knowledge and access sections state what the product may read, who may see what, the exact refusal wording, and where the audit trail sits.
• The incident section keeps the severity ladder, the first-hour steps, and the comms owner readable while someone is being paged, and every incident leaves eval cases behind.
• The team section guarantees coverage: every row has a person, and every person has a stand-in.
• The operating review is a standing session with four items: the dashboards, the gate log, incidents with their eval cases, and spend against the Inference Budget.
• One person signs the page, and the review date goes on the calendar before the meeting ends: the fillable Operations Charter leaves with you.

That closes Running an AI Organization. You put a name on every behavior, a gate on every change, rules on what the product reads and shows, a rehearsed answer for the public bad day, and a plan for the team that runs it, and the signed page in your hands is the one your own head of engineering can open before she ever has to ask.

Sources

• ISO/IEC (2023). ISO/IEC 42001:2023, Information technology, Artificial intelligence, Management system. Published December 2023; clause 9.3 requires management review of the AI management system at planned intervals.
• Anthropic (2025). Anthropic achieves ISO 42001 certification for responsible AI. Announcement of accredited certification, January 2025.
• Beyer, B., Jones, C., Petoff, J., and Murphy, N. R., editors (2016). Site Reliability Engineering: How Google Runs Production Systems. O'Reilly Media. Chapter 31 on the weekly production meeting (thirty to sixty minutes, a chair, a standing agenda) and Appendix F, example production meeting minutes.