The Whole of PyTorch on One Page
This series maps the entire PyTorch framework, from the Python API down to the kernels and hardware. Part 0 provides a high-level overview, revealing that the Python layer is tiny compared to the 235 MB of compiled C++ underneath. It introduces the dispatcher, kernels, memory management, and the two clocks (CPU and GPU), promising a floor-by-floor exploration in subsequent parts.
The Python you write all day is the smallest layer of it.