TokenTown: A Visual City Explaining How LLMs Work
TokenTown: A visual way to understand how LLMs work
I built TokenTown, an interactive isometric city where every district represents a stage of a transformer language model. As you type a prompt, a convoy carries the hidden state through tokenization, attention, and feed-forward layers, visualizing the actual math behind LLMs. While the weights are random for demonstration, the core mechanisms like KV cache and softmax are genuinely computed live in your browser to show exactly how these models generate text.
Treat the text this city writes as scenery; treat the mechanism as the lesson.
- foltik
I read through the explanations and they’re all typical hand wavy LLM prose, full of jargon, and random tangential details. If you’re going to publicly present this as a learning tool, it would be great if you spent some actual human brain cycles writing and polishing it by hand, rather than just chucking the first thing that claude spits out over the fence. I can pretty much guess the prompt you used and probably get the same thing.
- GCUMstlyHarmls
"This is not a loop in code: it is a stack of distinct blocks"
Proceeds to literally loop around 6 times.
- Hugada
I think you need to know a lot to understand this, so perhaps it should mention what you should already know in order for it to be comprehensible.