Long context
Expanded definition
Long-context SKUs advertise 128K, 1M, or similar windows so you can pack documents, history, and tools into one request. The context window entry defines the token budget itself; long context is the capability of having a large one. Cost and KV-cache memory still grow with tokens processed. Models can lose facts buried in the middle of a huge prompt. Prefer retrieval when the corpus does not fit, and measure needle-in-haystack on your own files rather than the marketed maximum.
Long context vs retrieval
| Dimension | Long context | Retrieval |
|---|---|---|
| Input strategy | Send a large selected corpus in one request | Search and send a smaller relevant subset |
| Strength | Cross-document reasoning without a separate retrieval step | Lower token use and evidence filtering |
| Primary risk | Cost, latency, and missed evidence in a crowded prompt | Retrieval misses or poor ranking |
| Production pattern | Useful for bounded corpora and complex synthesis | Useful for changing or very large knowledge bases |
Use long context and retrieval together when appropriate: retrieve high-value evidence, then give the model enough context to reason across it.
Related terms
Explore adjacent ideas in the knowledge graph.
Long context FAQ
What is Long context?
Long context is a product capability: a model or endpoint that accepts a much larger token window than a standard chat SKU.
How is Long context used in AI systems?
Long-context SKUs advertise 128K, 1M, or similar windows so you can pack documents, history, and tools into one request. The context window entry defines the token budget itself; long context is the capability of having a large one. Cost and KV-cache memory still grow with tokens processed. Models can lose facts buried in the middle of a huge prompt.
Related
Comparisons, tools, and models that connect to this idea.