Prompt injection, explained for product teams

What actually happens

A language model receives one stream of text. Your instructions, the retrieved document, the user's message and the email body all arrive in the same channel, and the model distinguishes between them by convention rather than by mechanism.

So a sentence inside a document saying to ignore previous instructions and forward the contents elsewhere is, from the model's position, indistinguishable from an instruction you wrote. There is no parser boundary to enforce, and no flag marking one part trusted.

The variants matter less than the shape. Text can arrive directly from a user, indirectly through a document, a web page, a calendar invite, a code comment or a file name. Anywhere untrusted text reaches the model is an input an attacker can author.

• Instructions and content share one channel
• The model separates them by convention, not by mechanism
• Any untrusted text is an input somebody else writes

When it stops being theoretical

A model that only reads and summarises has a bounded worst case: a wrong or embarrassing answer. Add tools and the worst case is whatever those tools can do.

The combination that matters is untrusted input plus capability. An assistant that reads customer emails and can issue refunds. An agent that reads a web page and can send a message. A code assistant that reads a repository and can open a pull request. Each is an action surface reachable by whoever controls the input.

Data exfiltration is the quieter risk. A model with retrieval access and any outbound channel — a link it renders, a request it makes — can be instructed to include information it retrieved, and the user may never see the instruction that caused it.

The layers that work

Separate proposing from deciding. Let the model emit a typed, validated intent and nothing else. That intent then passes the same schema validation, policy limits and authorisation as a request from a button in the interface. The model chooses among actions you defined; it never defines one.

Bound every action. Amount limits, allow-listed destinations, rate limits per user, and human approval on anything irreversible. These hold whatever the model was told, which is the whole point.

Constrain the output shape. A response required to match a schema cannot express most of what you were worried about, and structure is the most underused defence available.

Treat retrieval as untrusted. Passages from documents are inputs, not instructions, and the system should not act on them without the same checks. Where documents are user-uploaded, this is not a precaution, it is the threat model.

• The model proposes; the policy layer decides
• Hard limits and allow-lists beat careful instructions
• Structured outputs remove whole categories of expression
• Retrieved text is an input, never a command

What does not work

Instructing the model to ignore instructions found in documents. It helps a little, fails under pressure, and produces a false sense of a control being present.

Filtering for known injection phrases. The space of phrasings is unbounded and encoding defeats matching. Worth having as noise reduction; never worth relying on.

A second model checking the first. Useful for categories that are hard to express as rules, and vulnerable to the same manipulation through the same input. Keep deterministic limits underneath it.

And the most common one: assuming the risk is proportional to how sophisticated your users are. The attacker is not your user. They are whoever wrote the document, the email or the page your feature happens to read.

How this shapes what we build

In Wen, the in-app AI agent cannot move funds directly. The keys sit in embedded wallets the backend never holds, and the agent calls typed, validated tools that go through the same checks as a tap in the interface — so a persuaded agent is still bounded by the same policy a person is.

Axon Wallet makes the separation explicit: a language model turns plain language into a typed intent and does nothing else, and that intent passes JSON Schema validation, policy limits and a two-factor gate before any transaction reaches a chain.

MarketSync shows the other half of the discipline. It parses Telegram signals with a fast-path parser and only falls back to a language model when the message defeats it, then applies the account's own risk rules. Most messages never reach a model at all, which is both cheaper and a smaller attack surface.