# LLM Sampling: Temperature, Top-k, and Top-p

Canonical: https://llm-explained.vicoworks.com/en/learn/sampling-temperature/
Language: en
Publisher: Vicoworks (https://vicoworks.com)
Updated: 2026-10-25

The final-position representation passes through final LayerNorm and a vocabulary projection tied to the embedding. Greedy selects the largest logit; sampling draws from the distribution.

After the last block, final LayerNorm and the LM head map the final-position representation to vocabulary logits. Temperature and filters change the distribution before a token is selected.

## How it works
### Final LayerNorm
Final normalization follows the two demo blocks. Only the final position is used for the next token.

### Vocabulary projection
Output weights share the embedding matrix. Logits are raw scores for 16 tokens rather than percentages.

### Probability distribution
Positive temperature changes distribution shape. Temperature 0 uses argmax without dividing by zero.

### Select one token
Greedy decoding is deterministic; sampling may choose different outputs. High probability does not guarantee factual correctness.

## What can you observe in the lab?
Bars represent candidate tokens. Selecting one adds a token to context; the model does not generate a sentence all at once.

## Reference scale and simulation limits
Visuals use illustrative flows and fixtures. GPT-3 175B has 96 blocks, 96 heads per block, width 12,288, and a 2,048-token context. Its paper alternates dense and locally banded sparse attention; this lab examines causal dense attention. The numerical workbench uses a toy model with two blocks, two heads, width 8, and a vocabulary of 16. GPT-3 weights are not loaded, and presets do not generate answers from a trained language model.

## Primary sources
- [GPT-3 · Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165)
- [GPT-2 · reference implementation](https://github.com/openai/gpt-2/blob/master/src/model.py)
- [Attention Is All You Need](https://arxiv.org/abs/1706.03762)
- [Word representations · Mikolov et al.](https://arxiv.org/abs/1301.3781)
- [KV cache · Hugging Face](https://huggingface.co/docs/transformers/main/cache_explanation)