You are twenty minutes into the weekly product review when the support lead reads a transcript aloud. Your assistant has told a customer that refunds on annual plans clear "within 24 hours," a promise that appears nowhere in your policy, and the customer is now quoting it back in a billing dispute. You open the system prompt to find where the line came from, and the file history answers: the refund section was rewritten four months ago by an engineer who has since left the company. The quality dashboard that should have caught the drift shows a last-viewed date older than his exit interview. Everyone at the table agrees the answer is wrong, and nobody can say whose job it was to notice.
Nothing in that scene is a model failure. The model produced exactly what its prompt told it to produce and the dashboard recorded what it was built to record; the failure lives in the gaps between people: a change nobody reviewed, a signal nobody watched, a behavior nobody owned. This chapter opens the part of The Frontier about running an AI product, and its subject is that gap: the operating layer, the standing decisions that no model can make for you.
What actually failed at Klarna
An operating-layer failure can happen at a company whose AI genuinely works, which is what makes Klarna's support story worth dissecting.
- The announcement. In early 2024, Klarna reported that its OpenAI-powered assistant had handled 2.3 million conversations in its first month, two-thirds of all support chats, work the company equated to 700 full-time agents. By Klarna's numbers, resolution times fell from 11 minutes to under 2, satisfaction scores matched human agents, and the assistant was projected to add $40 million in profit that year.
- The doubling down. Late that year, CEO Sebastian Siemiatkowski told Bloomberg the company had stopped hiring about a year earlier and had let headcount fall from roughly 4,500 to 3,500 through attrition, saying "I am of the opinion that AI can already do all of the jobs that we as humans do."
- The correction. In mid 2025 the same CEO told the same outlet the drive for savings had gone too far: cost had been "a too predominant evaluation factor," with the result that "what you end up having is lower quality." Klarna began recruiting human agents again, promised customers could always reach a person, and called "investing in the quality of the human support" the way of the future.
Read that arc as an AI story and the moral is muddled, because the capability never collapsed: the assistant that was worth $40 million in 2024 was still resolving chats in 2025. Read it as an operations story and it comes into focus. Cost had an owner with real authority, the CEO himself, while quality had nobody of comparable standing; that is what "a too predominant evaluation factor" means in plain terms. Nobody's job was to review what two-thirds of customer conversations sounded like and force the trade-off back onto the table, so the correction came through the most expensive review loop a company has: the chief executive reversing himself in public.
A model does not run itself: every shipped AI product carries an operating layer of decisions only your organization can make, and failures that look like model failures are usually decisions nobody was assigned to make.
The recurring jobs in the operating layer
Your delivery loop already exists: you ship a behavior, you measure it with the instruments from Production signals: evals after the ship, and you improve it. The operating layer is the set of standing jobs wrapped around that loop, each one recurring for as long as the product speaks. Until you assign them, each job is done by whoever on your team happens to be nearest when it comes up, which is how the opening scene happens.
- Own the behavior. Every distinct thing the product does in front of users carries one accountable name, so the question "who approved this answer" never goes unanswered; writing the names down is where Ownership: one name on every model behavior begins.
- Release the change. A prompt edit can move behavior as far as a code deploy moves features, so it gets the same review, testing, staged rollout, and rollback. That discipline is change control, built in Change control: ship prompt and model changes like releases.
- Govern the knowledge and the access. Someone decides what the product is allowed to read and what each requester is allowed to see, the ground of Govern the knowledge: what your product is allowed to read and Govern access and safety: decide who may see what.
- Answer for it in public. When the product says the wrong thing to a customer, the response is rehearsed rather than improvised, which is what Incident response: when your product says the wrong thing in public prepares.
- Build the team. All of the above has to live in actual people with the right skills, the subject of Build the team: hiring for AI and raising the org's bar.
This is the principle from Supervision: keep a human in charge of the agent applied to an organization instead of a screen: the product still needs a boss, and past a certain size the boss is a set of assigned jobs rather than one attentive person.
The objection: we are three people, not an enterprise
The reasonable pushback is that everything above sounds like the machinery of a thousand-person company: sign-offs, review boards, a RACI matrix (the enterprise grid of who is responsible, accountable, consulted, and informed). You are three people shipping weekly, alive because you carry none of that weight, and bolting enterprise process onto a team that small is a real way to die.
But the layer is made of decisions, not headcount, and every decision in it already has an answer at your company today, assigned by default rather than by choice:
- The owner of the system prompt is whoever edited it last.
- The release process is whatever that person felt like testing.
- The knowledge policy is whatever the ingestion job happened to collect.
- The incident plan is improvisation.
You are not choosing whether to have an operating layer, only whether to write it down while that is cheap.
A three-person team can state the whole layer on one page, one name and one sentence per job, and the closing chapter of this part, Write your Operations Charter and run the operating review, produces that page along with the short recurring meeting, the operating review, that keeps it true. Klarna's gap was never headcount either: thousands of employees were on payroll while quality went unowned.
The operating layer is a set of decisions, not a headcount: a small team writes every one of them on a single page, and the alternative is not "no process" but unwritten process assigned by accident.
Try it now
This drill takes about fifteen minutes, needs nothing but a blank page, and produces the artifact the rest of the part builds on.
List every place your product speaks or acts. Write one row for each spot where model output reaches a user or a system: chat replies, generated emails, summaries, tool calls that move money or data, anything a customer could quote back at you.
List every input those behaviors depend on. Add rows for the system prompt, the retrieval corpus, the eval suite, the access rules, and the model version with its settings, because whoever controls an input controls the behavior downstream of it.
Write one name beside each row. Write a person rather than a team: the individual who can change that thing today and who answers when it misbehaves. If two people both qualify, pick the one who would be paged.
Mark the blanks honestly. A row with no name is what this drill exists to find, not a formatting problem. Three rows from a real inventory look like this:
| Where it speaks or acts | What it depends on | Owner today |
|---|---|---|
| Support chat replies | System prompt, policy corpus | Priya (product) |
| Weekly digest email | Summary prompt, usage data | no name since the handover |
| Refund tool call | Access rules, approval threshold | Tomas (engineering) |
Keep the page. It is your ownership inventory, and the next chapter's drill starts by turning its blanks into names.
Chapter Summary
- A shipped AI product carries an operating layer: standing decisions only your organization can make, wrapped around the loop of shipping, measuring, and improving.
- Klarna's assistant handled two-thirds of support chats in its first month and was projected to add $40 million in profit; the capability was real and stayed real.
- The failure was an operating decision: cost had the CEO's full attention while quality had no owner of comparable standing.
- Because no cheaper review loop existed, the correction arrived as the CEO reversing course in public more than a year later.
- Failures that look like model failures are usually decisions nobody was assigned to make.
- The recurring jobs: own each behavior, release changes with discipline, govern what the product reads and who may see what, answer for it in public, and build the team.
- The layer is decisions, not headcount; a three-person team writes it on one page, and the alternative is unwritten process assigned by accident.
- A dashboard is not an owner; every signal needs a person with the authority to act on it.
- Your ownership inventory, honest blanks included, is the artifact this chapter leaves behind.
- Next, Ownership: one name on every model behavior turns those blanks into names.
Sources
- Klarna (2024). "Klarna AI assistant handles two-thirds of customer service chats in its first month." Press release, February 27, 2024.
- OpenAI (2024). "Klarna's AI assistant does the work of 700 full-time agents." Customer story.
- Bloomberg (2024). "Klarna Stopped All Hiring a Year Ago to Replace Workers With AI, CEO Says." Interview with CEO Sebastian Siemiatkowski, December 12, 2024.
- Bloomberg (2025). "Klarna Turns From AI to Real Person Customer Service." Interview with CEO Sebastian Siemiatkowski, May 2025; the cost and quality quotes are from this coverage (last verified July 2026).