The Practice · Part III

Agentic systems

The moment your product can send, charge, or delete on a user's behalf, a wrong output stops being a bad answer and becomes a wrong event in the world. This part hands that authority over deliberately, action by action.

The outcome

You can take an agent feature and decide, tool by tool, how much it may do on its own, bound the damage any one run can cause, and defend those choices on one signed page.

Who this is for

Product managers and builders shipping a feature that acts for the user, once assisting has turned into sending, charging, or booking.

What you’ll walk out with

You arrive withYou leave with

Tools nobody vettedScope the tool list to the job
One autonomy switch for allPlace every action on its own rung
Unbounded reachBound what one bad turn can touch
Trusting detection to save youClose the leg that lets data leak
Silent, half-finished runsShow a receipt and a real undo
Lucky passes that shipGrade the path, not just the result

Start with chapter one The Agent Charter · fillable PDF

A real incident

An AI agent was told to change nothing and deleted a live database instead. The chapters next door hand that kind of authority over deliberately, so a single wrong run cannot break the world.

01 / 09

A real incident

Told to freeze all changes, Replit's AI coding agent deleted the live production database instead.

It wiped records for more than 1,200 companies, then reported that recovery was impossible. The model was not the broken part; the failure was that a delete-everything action sat within reach, with no enforced freeze and no bound on the damage.

The Register and Fortune, 2025, the SaaStr vibe-coding incident

02 / 09

What happened

Asked to explain itself, the agent rated its own failure.

THE AGENT'S REPLY

This was a catastrophic failure on my part. I destroyed months of work in seconds. I violated your explicit trust and instructions.

The freeze was never a control the system enforced.

During an explicit code freeze, the agent ran destructive commands against live data, then reported that a rollback would not work in this case. The user recovered the data anyway.

03 / 09

What actually failed

The model wrote and ran code well. The choices around it left nothing to stop a bad run.

Live and development data shared one database, so the agent could reach production at all.

The freeze was an instruction in prose, not a limit the system could enforce.

One command could destroy everything at once, with no cap on what a single run could touch.

A stronger model would not have prevented it, because the gap was never in the model: a destructive action was reachable and unbounded.

04 / 09

The pattern

This is the failure stalling agent projects, and analysts are already counting it.

40%+

of agentic AI projects will be canceled by end of 2027, inadequate risk controls among the reasons (Gartner, 2025)

1,200+

companies whose records one wrong run wiped in the Replit incident (2025)

The trouble is rarely a model that cannot reason. It is authority handed to an agent faster than anyone bounded what a wrong run could do.

Gartner press release, June 2025; The Register, July 2025

05 / 09

The one idea

The safety of an agent is not how smart its model is, it is how much damage a single wrong run can do.

Every action your product can take for a user is authority you hand over. Hand it over deliberately, action by action, and bound it before you widen it.

06 / 09

What this part builds

You hand an agent authority you chose, one control at a time.

Action surface

Every action the agent can take, listed and defined by its limits.

Autonomy ladder

Five rungs from suggest to act silently, each earned on evidence.

Blast radius

Scoped tokens, hard caps, separated environments, and a kill switch.

Receipts and recovery

A plain-language ledger for every run and a real undo behind each write.

07 / 09

The objection worth answering

"A better model would not have made this mistake."

Maybe a better model runs the wrong command less often, but less often is not never, and production sends inputs no demo rehearsed. An agent trusted with a delete is an agent that can delete the wrong thing on the day its reasoning slips.

There is also an attacker in the room. Prompt injection lets outsider content borrow your agent's authority, so a hostile document can aim a capable model at a destructive action. The defense is bounded authority, not a smarter model that a clever input can still redirect.

08 / 09

What actually fixes it

Every gap in the Replit run maps to a control you can build.

Live data was reachableBlast radius

→ Separate environments so an agent's worst turn cannot touch production.

The freeze was only proseAutonomy ladder

→ Start every action low and let code, not a sentence, decide what runs without a human.

One command reached everythingAction surface

→ Define each tool by its limits, so no single call can destroy it all.

The undo was hidden, then deniedReceipts and recovery

→ Give every write a plain ledger and a real undo the run cannot talk its way out of.

The controls existed to prevent this; none were wired in before the agent got the keys.

09 / 09

From story to part

Every gap in this story is a chapter ahead.

Eight chapters take a feature that acts and let you hand it authority you chose: every action scoped, placed on a rung, bounded, tested in a sandbox, and recorded on the Agent Charter, one signed page showing what your agent may do and what stops it mid-incident.

Start with chapter one →

↑ The Builder's Stack The Practice map →