Prompt injection

Prompt injection is an attack or failure mode where untrusted text tries to override system instructions or steer a model into unsafe behavior.

Prompt injection can appear in user messages, retrieved documents, web pages, emails, or tool outputs. Defenses include separating trusted instructions from untrusted content, limiting tool permissions, quoting retrieved text, using allowlists, adding policy checks, and monitoring for suspicious instructions. It is especially important in RAG and agent systems.