Your CPU at 100% Is Mostly Waiting on Memory
It's the Memory Latency, Stupid
A 100% CPU reading doesn't mean the processor is busy computing—often it's stalled on memory. A modern core can execute 4–8 instructions per nanosecond, while a DRAM access takes 50–100ns, so a single RAM fetch can cost 400–800 instructions of waiting. The Pivot team demonstrates this with a naive GROUP BY on ClickBench's 99M-row hits dataset: 50% of CPU time was spent stalled on memory and only 16% retiring operations. Radix partitioning into 4,096 pieces shrinks the working hash tables to fit in cache, cutting DRAM traffic from 4.5GB to 1.5GB and runtime from 3.6s to 2.7s.
This tells us that 50.1% of the time, the CPU couldn't do anything because it was stalling on memory! And only 16.1% of the time the CPU was actually retiring (aka doing) operations.
- boricj
I work with another team at work that uses lots of message passing, C++ virtual methods in deeply nested class hierarchies, large DOMs and off-the-shelf C++ STL containers... on a STM32MP1, which has an in-order Cortex-A7 and a pittance of cache.
On the other hand, in my team I wrote my custom C++ STL containers that allocate inline, SAX-style processing libraries and an object pool with links that use index+generations like 2000s video games... on a STM32H5 with less than a megabyte of RAM.
Mechanical sympathy feels like a lost art to people I demonstrate it to... and I had to demonstrate to the first team with benchmark numbers that I would not be the bottleneck between us by orders of magnitudes.
- happyPersonR
Shout out to the author, it seems coherent.
I like flamegraphs
I know folks use perf to help generate them
But in the future, now that Llm is here, get and agent to check assembly, PMUs and also see what you’re doing in assembly.
My 2c and happy coding to everyone :)
- drivebyhooting
I feel like the solution presented is just a clustered hashmap or “locality sensitive” if you’d like it.
Still not orders of magnitude better.
- brcmthrowaway
https://people.freebsd.org/~lstewart/articles/cpumemory.pdf
Has anyone rewritten this for 2026?