Embeddings

The vector representations language models use, with the linear algebra to reason about them. Defines an embedding as a learned vector (not readable named features); shows the embedding-matrix lookup that turns a token ID into a vector and how those values are learned by gradient descent; explains what distance in embedding space means and the two operations that measure it — the dot product and cosine similarity — with worked arithmetic. Then the ideas that matter in practice: why vector arithmetic sometimes reveals concepts (and why modern models make it messier), the difference between internal token embeddings and dedicated text embeddings, what makes a representation contextual, why individual dimensions usually aren't interpretable, how vector size and precision drive memory and cost, and how to inspect embeddings without being fooled by a 2-D plot.