What happens when a GPU reads memory
A deep dive into the hardware path of a single GPU memory read, following a vector-add kernel's global load from SASS instruction to DRAM. The author reverse-engineers the RTX 4090's memory system, revealing the costs of L1 hits, TLB misses, L2 hits, and DRAM accesses, and explaining the roles of the coalescer, L1 cache, TLB, crossbar, L2 slices, and memory controller.
One LDG.E asks for four bytes in each of 32 lanes; serving it takes four 32-byte sectors, one cache line, one address translation, a crossbar crossing, one of thirty-six L2 slices, and, when it misses everywhere, an activate and four column reads at a DRAM chip.