In June 2026, 404 Media reported (opens in a new tab) that attackers had used Meta's own AI support chatbot to take over high-profile Instagram accounts, including a US Space Force handle and the inactive Obama-era White House account. They didn't break anything. They asked.
The pattern is older than the iPhone. It's called the confused deputy, and it's been documented in security literature since 1988. Every AI-equipped SaaS app, including ours, has to understand it.
What a confused deputy actually is
Norm Hardy described the original case in a 1988 paper. A trusted program, the deputy, holds privileges that some other party, the requester, doesn't have. The requester asks the deputy to perform an action. The deputy performs it using its own authority rather than checking whether the requester was entitled to ask.
The classic example involves a compiler. The compiler has permission to write to a billing log. A user wants to overwrite that billing log. The user asks the compiler to write its output to the billing log. The compiler, holding write access to the disk, does what it's told. It doesn't ask whether the user was permitted to write there. It just writes.
The privilege didn't leak. The deputy used it for someone who wasn't entitled to invoke it.
AI assistants are confused deputies by default. The assistant holds privileges the human user typically doesn't: it can read internal documentation, query support systems, escalate cases, sometimes execute account changes. A user talks to it in natural language. The assistant tries to be helpful.
Why AI is a perfect modern instance
In a 1988 deputy, the privileges and the operations were narrow. A compiler wrote files. A printer queued jobs. The attack surface was small and the deputy's logic was inspectable.
Modern AI assistants invert all three of those properties.
The privileges are broad. A support assistant for a major platform often has read access to user records, read access to internal documentation, and write access to action queues. Some have outbound mail privileges. Some have privileges to issue refunds.
The operations are open-ended. The assistant isn't restricted to "write file" or "queue job." It accepts free-form natural language and translates that into actions. The set of possible inputs is, in practice, infinite.
The logic isn't inspectable. The decision about whether a given request should be honoured is made by a language model on the basis of patterns the system's authors didn't write down. There's no source code you can read to know what the deputy will do next month, because the deputy is a probability distribution.
What seems to have happened at Meta
Based on the public reporting (opens in a new tab), attackers approached Meta's AI support chatbot, identified themselves as the legitimate owners of accounts they didn't own, and asked the assistant to take actions on those accounts. The assistant, by design, was helpful. The actions were taken.
We don't have the implementation details. We're not going to speculate on what's broken inside Meta. Their engineers face the same problem ours do, and the public reporting isn't enough to fairly assign root cause.
What we can do is name the pattern, because the pattern explains it.
The work of binding that assertion to a real identity, then bounding what actions the assistant can take on that identity's behalf, has to happen outside the conversational layer. If it happens inside, the model's helpfulness will route around it.
Three design principles that prevent this
There are dozens of ways to address confused-deputy risk in AI systems. Three of them are non-negotiable.
Identity binding
The AI agent acts on behalf of an authenticated user, not on behalf of whoever's currently talking to it. The authentication happens before the conversation starts, at the session layer, and the identity is held outside the model's context. The model doesn't decide who the user is. The session does.
If your AI assistant infers user identity from the conversation, you have built a confused deputy.
Capability separation
The assistant can only perform operations the authenticated user themselves can perform. If the user can't reset another user's password through the normal product surface, the AI assistant can't do it on their behalf either.
This sounds obvious. It's surprisingly hard. Most AI assistants are given a superset of user permissions for "convenience," with the expectation that they'll exercise that superset only when appropriate. Convenience plus probabilistic reasoning equals exploit surface.
Out-of-band confirmation on irreversible actions
Anything that can't be undone (account ownership transfer, irreversible delete, outbound communication to a customer's mailing list) requires confirmation through a channel the AI doesn't control. Usually that's an email to the user's verified address, or a notification in the product interface they actually own.
The point isn't to slow the assistant down. The point is to interrupt the deputy.
How Auraflow approached this
We don't run a customer-facing AI assistant. Our agent layer is for the merchant, operating their own store. But the same patterns apply, and we built the boundary in early because retrofitting it later is expensive.
The merchant authenticates to the Auraflow dashboard. The merchant's session, not the agent's reasoning, holds the identity. The agent can only take actions in categories the merchant has explicitly enabled. Sending email, deleting data, creating discount codes and pushing audiences always require merchant approval inside the dashboard, not inside the conversation.
We're not claiming this is novel. The 1988 paper is novel. Our job was to apply it before our product surface required it. The same posture shapes how we handle visitor data and what the CLV engine is allowed to do with it.
There are obvious next questions. What about supply-chain prompts? What about model-side attacks that try to subvert the boundary? We have answers, and we'll write about them separately. The principle layer is the right place to start.
Strong AI agents are about to be everywhere. The interesting question isn't how smart they are. It's what they're allowed to do, on whose behalf, and who's accountable when they do it on the wrong person's.