Long Context
Expanded definition
Long context refers to the maximum token window a model or endpoint can process in one request. The window may include system instructions, user input, retrieved passages, images represented as tokens, tool results, conversation history, and output tokens, depending on the provider. Maximum context is not the same as effective context. Models can lose important details in long inputs, especially when evidence is duplicated, contradictory, or poorly positioned. Context limits may also differ by endpoint, deployment, modality, account tier, or maximum output allocation. Long prompts increase cost, latency, and attention load. Retrieval, summarization, structured memory, and document selection remain useful even with a one-million-token window. Evaluate recall by position, citation accuracy, instruction retention, latency, and total cost on representative workloads.
Long context vs retrieval
| Dimension | Long context | Retrieval |
|---|---|---|
| Input strategy | Send a large selected corpus in one request | Search and send a smaller relevant subset |
| Strength | Cross-document reasoning without a separate retrieval step | Lower token use and evidence filtering |
| Primary risk | Cost, latency, and missed evidence in a crowded prompt | Retrieval misses or poor ranking |
| Production pattern | Useful for bounded corpora and complex synthesis | Useful for changing or very large knowledge bases |
Use long context and retrieval together when appropriate: retrieve high-value evidence, then give the model enough context to reason across it.
Related terms
Explore adjacent ideas in the knowledge graph.
Long Context FAQ
What is Long Context?
Long context is an LLM's ability to accept a large token window containing prompts, documents, conversation history, tool results, and generated output.
How is Long Context used in AI systems?
Long context refers to the maximum token window a model or endpoint can process in one request. The window may include system instructions, user input, retrieved passages, images represented as tokens, tool results, conversation history, and output tokens, depending on the provider. Maximum context is not the same as effective context. Models can lose important details in long inputs, especially...
Related
Comparisons, tools, and models that connect to this idea.