Safety
Prompt injection
Prompt injection is an attack or failure mode where untrusted text tries to override system instructions or steer a model into unsafe behavior.
Expanded definition
Prompt injection can appear in user messages, retrieved documents, web pages, emails, or tool outputs. Defenses include separating trusted instructions from untrusted content, limiting tool permissions, quoting retrieved text, using allowlists, adding policy checks, and monitoring for suspicious instructions. It is especially important in RAG and agent systems.
Related terms
Explore adjacent ideas in the knowledge graph.
Related
Comparisons, tools, and models that connect to this idea.