Gradient descent and backpropagation

How a single scalar loss produces coordinated updates to every one of a model's parameters. Defines a derivative and a gradient; introduces the gradient descent update rule and works one update by hand; explains backpropagation as the efficient algorithm — reverse-mode automatic differentiation — that computes the whole gradient in about the cost of three forward passes, including the exact softmax-cross-entropy gradient that closes the loop from *Loss and cross-entropy*; covers training batches and stochastic gradient descent; explains what optimizers like Adam and AdamW add beyond plain gradient descent; why the learning rate is the most consequential hyperparameter, with warmup and decay schedules; exploding and vanishing gradients and the architecture that prevents them; and how training is split across many machines.