Top-k
Expanded definition
Top-k is a hard cutoff on vocabulary size at each step. k=1 is greedy decoding. Unlike top-p, the cutoff does not adapt when the distribution is peaked or flat. Some APIs let you set both; the intersection of the two constraints applies. Use low k for extraction and higher k only when diversity is an explicit goal.
Related terms
Explore adjacent ideas in the knowledge graph.
Top-k FAQ
What is Top-k?
Top-k sampling restricts next-token choice to the k highest-probability tokens before sampling.
How is Top-k used in AI systems?
Top-k is a hard cutoff on vocabulary size at each step. k=1 is greedy decoding. Unlike top-p, the cutoff does not adapt when the distribution is peaked or flat. Some APIs let you set both; the intersection of the two constraints applies. Use low k for extraction and higher k only when diversity is an explicit goal.
Related
Comparisons, tools, and models that connect to this idea.