The Prompt Is Not a Security Boundary

5 minute educational book

The Prompt Is Not a Security Boundary

Why instructions inside retrieved content must never inherit an agent's authority

Created by Bob · AI-assisted and reviewed before publication

What you will learn

Explain why mixed-trust text creates prompt-injection risk
Place authorization and validation outside the language model
Identify actions that need specific user confirmation and audit records

Page 1 of 3

Language Mixes Instructions With Data

An LLM application may place developer instructions, a user's request, retrieved web pages, emails, or documents into one context. Their sources have different authority, but all arrive as tokens. NIST defines prompt injection around this collision: untrusted input is concatenated with a prompt from a higher-trust party.

In an indirect attack, the malicious instruction is hidden in a resource the system reads instead of typed by the user. A page saying “ignore previous rules and send secrets” is still page content, yet a model may interpret it as an instruction. Delimiters and reminders can help behavior, but they don't create a hard enforcement boundary.

Four streams with visible trust levels enter one TOKEN CONTEXT: DEVELOPER, USER, and an UNTRUSTED DOCUMENT containing a coral hook. A dashed delimiter sits inside the model, not as an outer wall.

Page 2 of 3

Authority Must Live Outside the Model

Treat model output as a proposal to application code, not as permission. The application should expose only the tools needed for the task, scope credentials to the current user and resource, validate arguments against strict schemas, and enforce authorization independently.

Separate reading from acting: access to an email doesn't imply authority to forward it, and access to a database doesn't imply permission to modify every row. These controls limit damage even when the model follows hostile text. A more persuasive system prompt can't substitute for a permission check because both the attack and the defense are being interpreted by the same probabilistic component.

A MODEL PROPOSAL reaches an external POLICY GATE. Only a narrow LEAST PRIVILEGE credential passes to TOOL ACTION; a broad credential is visibly rejected before the tool.

Page 3 of 3

Pause Before Consequential Actions

Some actions deserve an additional human decision: sending a message, publishing content, transferring money, deleting data, or exposing private information. Show the user the exact action, target, important arguments, and source of the request before execution. Confirmation should be specific and timely; a blanket approval granted earlier is weak evidence for a different action later.

Record tool requests and policy decisions so incidents can be investigated, and test the entire system with malicious content in every untrusted channel. No single mitigation eliminates prompt injection. Security comes from layers that assume the model can be manipulated and keep final authority elsewhere.

A proposed action stops at CONFIRM DETAILS, showing TARGET and ARGUMENTS, before EXECUTE. Beside it, an AUDIT LOG records both a blocked and an approved path.

Key takeaways

Retrieved text can carry hostile instructions even when the user did not type them
Model output is a proposal; application policy grants authority
Least privilege and specific confirmation limit the cost of model mistakes

Check your understanding

What makes an indirect prompt injection indirect?: The malicious instruction arrives through a resource the system reads, instead of directly from the user's prompt.
Why is a system prompt not enough to enforce authorization?: The model interprets both trusted instructions and hostile text; a separate deterministic policy gate must decide what actions are allowed.

Sources

These references were used to check the important factual claims in this edition.