Large Language Models
You'll be able to
- Understand what an LLM is trained to do
- Explain 'next token prediction' simply
- Appreciate the scale of modern models
A large language model (LLM) is trained on enormous amounts of text to do one simple-sounding thing: predict the next token (a word fragment) given the tokens before it. From billions of examples it internalises grammar, facts, reasoning patterns, and even style.
At generation time, the model predicts the next token, adds it, and repeats. This loop is remarkably powerful — coherent essays, working code, and helpful explanations all emerge from iterated token prediction.
prompt = "The capital of France is"
tokens = tokenize(prompt)
while not done:
next = model(tokens) # predict distribution over next token
token = sample(next) # pick one (greedy or random)
tokens.append(token) # feed it back inChallenge
Next-token thinking
Complete this sentence two different ways and reflect on how many reasonable continuations exist: 'After the storm, the city...'
Knowledge Check
LLMs
At its core, an LLM is trained to:
LLMs can generate fluent text but can also produce confident-sounding errors.
Answer all questions to submit.