Prompt Injection

Prompt injection is an attack where untrusted input — a document, a webpage, a user message — contains instructions crafted to override a model's original system prompt or task, hijacking its behavior. It matters most once a model can retrieve untrusted content or call tools, since a successful injection can turn a summarization task into an unauthorized action. Defending against it takes input/output evals and guardrails, not just a stricter system prompt, because the system prompt is exactly what's being attacked.

← Back to the full glossary