How a language model writes one word at a time
Process map by Brian. 13 steps, 2 layers. Tags: ai, tech.
- You: You type a prompt
- The tokenizer: Split it into tokens (word pieces)
- The neural network: Turn the tokens into numbers (embeddings)
- The neural network: Attention layers decide which earlier words matter most
- The neural network: Compute a probability for every possible next token
- The sampler: Pick one (temperature controls how adventurous)
- You: Is it a stop token?
- You: The answer is complete
- The neural network: Add the token to the text and repeat