NEUROLAB / Created by VICOWORKS · 2026-10-25
LLM Sampling: Temperature, Top-k, and Top-p
The final-position representation passes through final LayerNorm and a vocabulary projection tied to the embedding. Greedy selects the largest logit; sampling draws from the distribution.
Try it in the 3D lab →How it works
After the last block, final LayerNorm and the LM head map the final-position representation to vocabulary logits. Temperature and filters change the distribution before a token is selected.
Final LayerNorm
Final normalization follows the two demo blocks. Only the final position is used for the next token.
Vocabulary projection
Output weights share the embedding matrix. Logits are raw scores for 16 tokens rather than percentages.
Probability distribution
Positive temperature changes distribution shape. Temperature 0 uses argmax without dividing by zero.
Select one token
Greedy decoding is deterministic; sampling may choose different outputs. High probability does not guarantee factual correctness.
What can you observe in the lab?
Bars represent candidate tokens. Selecting one adds a token to context; the model does not generate a sentence all at once.
The Scanner opens notes. The Token tracer selects token positions. The Head lens compares two demo heads. The Time controller pauses signals. The Calibrator changes experiment conditions before you predict and test the outcome.
What to remember
Visuals use illustrative flows and fixtures. GPT-3 175B has 96 blocks, 96 heads per block, width 12,288, and a 2,048-token context. Its paper alternates dense and locally banded sparse attention; this lab examines causal dense attention. The numerical workbench uses a toy model with two blocks, two heads, width 8, and a vocabulary of 16. GPT-3 weights are not loaded, and presets do not generate answers from a trained language model.
Primary sources
- GPT-3 · Language Models are Few-Shot Learners
175B scale: 96 blocks, width 12,288, 96 heads, context 2,048; alternating dense and locally banded sparse attention.
- GPT-2 · reference implementation
Pre-norm, learned positions, GELU, residuals, and tied embeddings.
- Attention Is All You Need
Foundations of scaled dot-product and multi-head attention.
- Word representations · Mikolov et al.
Background on static word embeddings and geometry.
- KV cache · Hugging Face
Prefill, decoding, and K/V reuse.