Concept graph
Glossary
Short definitions with deeper context and cross-links to sibling terms.
Safety
attention
Attention weights how much each token should attend to others when producing representations.
Model Capabilities
Long Context
Long context is an LLM's ability to accept a large token window containing prompts, documents, conversation history, tool results, and generated output.
Model Architecture
Mixture of Experts
A mixture-of-experts model routes each token through a selected subset of specialized neural-network components instead of activating the full parameter set.
Product
softmax
Softmax converts a vector of logits into a probability distribution over classes or tokens.
Training
token
A token is the smallest unit a model consumes or generates; pricing and context limits are expressed in tokens.
Natural Language Processing
Tokenization
The process of converting text into smaller pieces, called tokens.
Training
top-k
Top-k sampling restricts each step to the k highest-probability tokens.
Data
top-p
Top-p (nucleus) sampling keeps the smallest set of tokens whose cumulative probability exceeds p.