Build intuition before
diving deeper.
A comprehensive reference for the concepts that appear throughout the lab. Written for beginners, designed for deep understanding.
Read a concept, then test it immediately.
Each idea here maps directly to a workflow in the Playground, Compare, or Visualizer page. The absolute fastest way to build an intuitive understanding of AI is to move seamlessly between reading an explanation and running an experiment.
Start
Read a short concept here.
Run
Try it out in a lab.
Reflect
Connect the result to the math.
Core Concepts
The fundamental building blocks of Large Language Models.
What is an LLM?
At its core, a Large Language Model (LLM) is an incredibly complex pattern-matching engine. It does not "think" or "know" facts in the human sense. Instead, it predicts the most statistically probable next token based on the massive amounts of text it digested during training.
Imagine playing autocomplete on your phone, but scaled up to billions of parameters. By seeing a sequence of tokens, the neural network calculates a probability distribution across its entire vocabulary to determine what comes next.
What is tokenization?
Models cannot read raw letters or strings. Before text enters the model, a Tokenizer chops it into smaller chunks called tokens. A token might be a single character, a prefix, or a whole word.
Most modern models use Byte-Pair Encoding (BPE), which merges frequently occurring characters into single tokens to save space. Notably, spaces are often attached to the beginning of words (like the Ġ token in GPT-2) so the model knows where words begin and end.
What are token IDs?
Once the text is chopped into tokens, each unique token is mapped to an integer called a Token ID. This mapping is defined by the model's fixed Vocabulary.
For example, the token " hello" might map to ID 31414. These IDs are then packed into mathematical grids called PyTorch Tensors. If we batch multiple prompts together, shorter sequences are padded with a special padding token (often ID 50256) so the tensor remains a perfect rectangle.
What is attention?
The secret sauce of modern LLMs is the Self-Attention Mechanism. As the model processes a sequence of tokens, attention allows it to weigh the importance of every single token against every other token in the context window.
This is how a model resolves ambiguity. If it reads "The bank of the river", attention heavily connects "bank" to "river", ignoring the financial definition of "bank". We also use an Attention Mask (1s and 0s) to tell the model to ignore artificial padding tokens.
What is sampling?
When the model predicts the next token, it produces a list of probabilities (logits). Sampling is the algorithm used to pick the winning token.
- Temperature: Controls randomness. Higher values flatten the probabilities, making unexpected tokens more likely.
- Top-K: Restricts the choice to only the top K most likely tokens, cutting off the long tail of weird words.
- Top-P: Restricts the choice to a dynamic pool of tokens whose combined probability reaches P (e.g. 0.9).
What is decoding?
Once a token ID is sampled, the generation loop isn't finished. That ID must be appended to the input sequence, and the entire sequence is fed back into the model to predict the next token. This is why it's called autoregressive generation.
Simultaneously, the new ID is passed back through the Tokenizer in reverse—a process called Decoding—to translate the number back into human-readable text on your screen.