Loss and cross-entropy

How next-token training turns a predicted probability distribution and the observed token into a single error number the optimizer can minimize. Defines a loss function (and how it differs from an evaluation metric); shows the one-hot next-token target; builds cross-entropy from its general form down to the single term that matters for language models, negative log-probability of the correct token; works the loss by hand; explains why the logarithm is the right choice (steep penalties, additive sequences, maximum likelihood); combines losses across a sequence with masking and reduction; covers teacher forcing and the training-vs-generation gap; connects cross-entropy to maximum-likelihood; defines perplexity and its comparability traps; and ends with why lower training loss can still mean a worse product, and how validation loss is used.