Preparing the 3D world…
Read the guide while loadingPreparing the 3D world…
Read the guide while loadingNEUROLAB / VICOWORKS
NeuroLab is a browser-based LLM learning laboratory by Vicoworks. Explore a first-person 3D world, use five tools, and test your understanding through six missions. This guide helps IT and data practitioners understand how large language models work, including decoder Transformer mechanisms underlying the GPT family.
Back to the lab ↑A model sees token IDs. Tokens can be words, word pieces, punctuation, or characters including a leading space.
An ID selects an embedding row. Adding a learned position vector encodes order. The ID itself is not a numerical measure of meaning.
Queries match patterns, Keys offer patterns to match, and Values carry information to mix. All three are learned projections of the same representation.
Query–Key scores are divided by the square root of head dimension. Future positions are masked, then softmax produces weights to mix Values.
Each block contains attention and an MLP. Two residual connections retain input paths. LayerNorm operates across each token's feature dimension.
The final-position representation passes through final LayerNorm and a vocabulary projection tied to the embedding. Greedy selects the largest logit; sampling draws from the distribution.
Previous Keys and Values can be reused at each layer. Only the new token produces new Query, Key, and Value vectors. The cache contains this sequence's activations rather than weights or shared user memory.
Vector arithmetic illustrates relationships in static word embeddings. This example is not the literal mechanism by which GPT answers, and the relationship is not universal.
Click the world to look around, walk with WASD, choose tools 1–5, and press E to use them. H opens Hint and F opens missions. On phones, use the joystick, drag to look, and tap Use. The journal offers Meaning, Numbers, and Code tabs. A 2D mode is available without WebGL.
The Scanner opens notes. The Token tracer selects token positions. The Head lens compares two demo heads. The Time controller pauses signals. The Calibrator changes experiment conditions before you predict and test the outcome.
This guide works without JavaScript. The 3D world and experiments require JavaScript; all processing happens in your browser.
Visuals use illustrative flows and fixtures. GPT-3 175B has 96 blocks, 96 heads per block, width 12,288, and a 2,048-token context. Its paper alternates dense and locally banded sparse attention; this lab examines causal dense attention. The numerical workbench uses a toy model with two blocks, two heads, width 8, and a vocabulary of 16. GPT-3 weights are not loaded, and presets do not generate answers from a trained language model.
A large language model learns language patterns from data. An autoregressive decoder predicts a next-token distribution from context, selects a token, and repeats.
The lab is free and runs client-side. No API key, account, or inference server is required. Local presets and a toy model support learning.
Attention weights describe a mixture of Values within one head and step. They are not a complete explanation of a model's decision or a guarantee of factual correctness.
Training updates weights through loss, backpropagation, and an optimizer. Inference keeps weights fixed while producing activations and predicting the next token.
175B scale: 96 blocks, width 12,288, 96 heads, context 2,048; alternating dense and locally banded sparse attention.
Pre-norm, learned positions, GELU, residuals, and tied embeddings.
Foundations of scaled dot-product and multi-head attention.
Background on static word embeddings and geometry.
Prefill, decoding, and K/V reuse.