Lesson 2 · Natural Language Processing
Tokenization & Embeddings
8 min
You'll be able to
- Understand how text is split into tokens
- Explain why models need embeddings
- Build intuition for semantic vectors
Models cannot read characters directly, so text is first tokenized — split into words or subword pieces — then each token is mapped to a vector via an embedding. Embeddings place similar words near each other in a high-dimensional space.
'King' and 'Queen' end up near each other, and relationships like gender can appear as consistent vector offsets — a hint of the structure models learn.
Challenge
Vector intuition
Write down three words that should be near 'ocean' in embedding space, and three that should be far away.
Knowledge Check
Tokenization & embeddings
Embeddings represent words as vectors where similar words are close together.
Tokenization is the process of:
Answer all questions to submit.