Week 5 deep learning with transformers

I. Text to vectors

I. a: Tokenization

I. b: Embeddings

I. c: Word Vectors

II. Language models

II. a: Language modelling

Autoregressive factorization

Other factorizations

Distribution of the possible next tokens

Language models learn contextual representations

Sampling