How a neural net reads a sentence it has never seen
An intuition for how neural nets can understand text
A four-step intuition ladder for how neural networks understand text: a single neuron acting like an if-statement, a bag-of-words network that learns rules instead of memorizing lists, embeddings that give each word a meaning profile learned through fill-in-the-blank, and emergent concepts such as the sentiment neuron OpenAI found in 2017 in a model trained only to predict the next character. Each insight comes with a short video, a written explanation, and practice.
Words that fit the same blanks drift together, so tiger lands next to owl even if the two never shared a sentence in the training data.