Govern access and safety: decide who may see what

Nothing in that screenshot is a security bug: the permissions did what they said and the assistant obeyed them, yet a person read a plan he was never meant to read, because "may technically open this file" and "should learn this from the assistant" were never the same question. Who may see what is a product decision, and running it takes three pieces: access rules written as product policy, refusals that do not leak, and an audit trail that shows what happened.

Access control becomes behavior

In classic software, access control is plumbing: no permission, a 403 error, and the page is not there. In an AI product, the control moves into the answer itself. "What is your refund policy" should draw on public help pages for an anonymous visitor, add the terms of the asker's own plan for a signed-in customer, and include the internal exceptions list for a support agent.

In an AI product, access control is behavior: the same question earns different answers depending on who asks, so who-may-see-what is a product rule, not an infrastructure setting.

The rule has one workable enforcement point: retrieval. Trim what may be read for a requester before the model reads it, never after, because a passage that enters the context influences the answer even when its words never appear. Each tier needs two decisions: what retrieval may read on its behalf, and what the answer may reveal, down to whether it may confirm a thing exists. Stacked by tier, the policy looks like this:

Written as product policy for a support assistant, the rules fit on one screen:

• A customer sees their own account's data (tickets, invoices, plan) plus the public help center, and nothing else.
• A support agent sees the accounts in their queue and the internal knowledge base, not the whole customer book.
• Internal notes on a ticket are readable by staff and never quoted to the customer they discuss.
• Nobody sees another tenant's content (a different customer organization on your product), and the assistant may not confirm that another tenant exists.
• An admin who can open everything still gets a refusal on payroll and legal folders: operating the system and reading its most sensitive files are different jobs.

The Copilot rollouts: permissions that were correct and still wrong

Through 2024, enterprises rolling out Microsoft 365 Copilot hit the opening scene at scale. Copilot answers from whatever the signed-in employee can technically reach across SharePoint, OneDrive, and mail, and in tenant after tenant that reach included sites opened years earlier to defaults like "Everyone except external users." Employees found pay information, HR documents, and in reported cases executives' emails, all technically permissioned to them and none ever locked down, because before Copilot nobody could find it. In a 2024 Gartner survey of 132 IT leaders, 40 percent said oversharing concerns had delayed their rollout by three months or more.

Microsoft's fix barely touched the model. Restricted SharePoint Search, released mid-2024, let administrators limit Copilot and organization-wide search to an allow list of up to 100 reviewed sites; Microsoft's documentation calls it a temporary measure that "isn't a security boundary," there to buy time while permissions get audited. The durable tooling arrived at the Ignite conference that November:

• SharePoint Advanced Management, folded into Copilot subscriptions, to find overshared sites,
• Purview assessments to report on them,
• and a deployment blueprint on Microsoft Learn covering permission review from pilot through operation.

That is an access-governance process: inventory what can be read, decide what should be, keep checking after launch. The permissions were technically correct and behaviorally wrong; the fix was organizational.

"Our permission system already handles this"

The objection is half right. A permission system answers "can": which identities may open which resources, and your assistant must inherit that layer, never bypass it. But the Copilot story is "can" enforced perfectly, and salary data still landed in a chat window. Permissions accumulate one locally sensible grant at a time, obscurity absorbs the gap between what people could open and what anyone meant them to read, and retrieval deletes the obscurity. Product policy answers "should," and nothing in the permission stack ever will, so you keep both layers: the permission system as the floor, the who-may-see-what lines above it.

Refuse without confirming the secret exists

The rules also need wording for the moment someone asks above their tier. The instinctive refusal describes what it withholds: "I can't share details of the Northwind acquisition" politely confirms an acquisition called Northwind exists. A refusal that changes with what is behind the door is an answer an adversary can harvest one probing question at a time; Data: what flows in and what leaks out covers how answers, citations, and refusals become an extraction channel.

For the support assistant above, asked "what plan is Acme Corp on?", the safe refusal reads:

"I can only look up information for your own account. Is there something on your account I can help you with?"

The same sentence goes out whether Acme is your largest tenant or no customer at all.

A safe refusal declines without confirming the hidden thing exists: if the refusal reads differently when there is a secret than when there is nothing, it is leaking.

The audit trail: who asked, what was read, what was shown

The last piece is being able to say afterward exactly what your product revealed, and to whom. On the bad day (a pay figure quoted in chat, a customer claiming the assistant described another tenant's contract) the questions are always the same, so log their answers on every exchange:

• who asked, with their tier,
• the question as asked,
• every document retrieval read, not only those cited,
• what the answer showed, or which refusal fired,
• and the access-rules version in force at the time.

The version line matters because rules change, and a who-may-see-what change ships like any release, with the discipline from Change control: ship prompt and model changes like releases. The trail is also becoming a legal floor: the EU AI Act requires high-risk systems to log events automatically, with logs kept at least six months, and after the 2026 Digital Omnibus amendment its high-risk obligations phase in from December 2027; the vertical detail lives in Testing, evidence, and audit trails. It also turns "the assistant showed someone something" from archaeology into a lookup, the first thing whoever is on call opens in Incident response: when your product says the wrong thing in public.

Try it now

The drill takes about fifteen minutes and needs nothing from earlier chapters.

Choose the feature. Pick one place where your product answers with retrieved content: the assistant, the search box, or the drafting tool.

Write the who-may-see-what lines. Write three to five lines in the form above, each pairing a requester tier with what retrieval may read for it, including one line for what nobody sees and one thing the product may not confirm exists.

Write one refusal verbatim. Draft the exact sentence for a requester asking above their tier, worded to read the same whether the hidden thing exists or not; if a stranger could guess the secret from the wording, rewrite it.

State where the audit log lives. Write one line on where who-asked, what-was-read, and what-was-shown are recorded today; if the honest answer is "nowhere," write that, because it is the first gap to close.

Keep the page. These lines become the access section of the charter you assemble in Write your Operations Charter and run the operating review, and they extend the posture document from Write your Security Posture and ship defended if you built one.

Chapter Summary

• In an AI product, access control becomes behavior: the same question earns different answers depending on who asks.
• Enforce access in retrieval, before the model reads anything, because content that enters the context influences the answer even when it is never quoted.
• Write who-may-see-what as product lines, one per requester tier, each testable by logging in as that requester and asking.
• The Microsoft 365 Copilot oversharing episode showed permissions that were technically correct and behaviorally wrong: every check passed, and employees still found pay data and executives' emails.
• The fix there was governance, not a model change: inventory what can be read, decide what should be, and keep auditing after launch.
• Your permission system answers "can"; your access rules answer "should"; you need both layers.
• A safe refusal declines without confirming the hidden thing exists, in wording that stays identical whether there is a secret or not.
• Log who asked, what was retrieved, what was shown, and which version of the rules applied; regulation increasingly requires exactly this record.
• The day an answer goes wrong in public, that log is the first thing you will open, and Incident response: when your product says the wrong thing in public covers what happens next.

Sources

• Computerworld (2024). Microsoft moves to stop M365 Copilot from "oversharing" data. Covers the Ignite 2024 announcements and the 2024 Gartner survey of 132 IT leaders.
• The Register (August 2024) and Business Insider (November 2024). Reported Copilot oversharing cases: salary and payroll data, HR documents, and executives' emails, all reachable through inherited permissions.
• Microsoft Learn. Restricted SharePoint Search documentation: the temporary allow list of up to 100 sites and the statement that it "isn't a security boundary"; Microsoft is retiring the feature, with new enablement blocked from late July 2026 (last verified July 2026).
• Microsoft Learn. Microsoft 365 Copilot blueprint for oversharing: the pilot, deploy, operate guidance (last verified July 2026).
• Microsoft (2024). Introducing Restricted SharePoint Search, Microsoft 365 Copilot blog; general availability June 2024.
• European Union (2024). Regulation 2024/1689 (the AI Act), Articles 12 and 19: automatic event logging for high-risk systems and provider log retention of at least six months. The 2026 Digital Omnibus on AI postpones the high-risk obligations, originally phasing in from August 2026, to December 2, 2027 for stand-alone Annex III systems and August 2, 2028 for AI embedded in regulated products (last verified July 2026).