Context window
Expanded definition
The advertised context length is a hard cap, not usable working memory. System prompts, tool schemas, retrieved chunks, and the model's own output all share the budget. Long-context models can still lose track of buried facts; packing and position matter. Cost and latency usually grow with tokens processed, especially when the KV cache is large. Prefer retrieval over stuffing a whole corpus into the window.
Related terms
Explore adjacent ideas in the knowledge graph.
Context window FAQ
What is Context window?
A context window is the maximum number of tokens a model can consider at once, including instructions, history, tools, and output.
How is Context window used in AI systems?
The advertised context length is a hard cap, not usable working memory. System prompts, tool schemas, retrieved chunks, and the model's own output all share the budget. Long-context models can still lose track of buried facts; packing and position matter. Cost and latency usually grow with tokens processed, especially when the KV cache is large. Prefer retrieval over stuffing a whole corpus into...
Related
Comparisons, tools, and models that connect to this idea.