A Linux kernel module doubles FP/BF16 throughput on Nvidia RTX 30-series cards
GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards
This Linux kernel module unlocks the full FP/BF16 instruction issue rate on Nvidia GA102, GA103, GA104, and GA106 GPUs, matching their professional counterparts. It combines the CMP unlocker's V67/PLM technique with UEFI-style pre-driver timing, writing speed selectors and performing FLR before the unmodified NVIDIA driver binds the card. On a 3090 Ti, BF16 GEMM jumps from 87.5 to 157.4 TFLOPS, FP16 from 87.5 to 118.4, and TF32 from 42.6 to 82.6. Installation requires adding a kernel command-line option.
The effect of this unlock is that it doubles the instruction issuing speed of FP/BF16 FMA into FP32 operations on GA102, GA103, GA104 and GA106 cards.
- Zagitta
Hey this is my work, although the exploits etc aren't my ideas by any means.
This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.
Happy to answer questions!
- jacquesm
This one is real, but there are some caveats:
- you will need to run your card hotter than you might be comfortable with
- the real pay-off (on some cards, apparently, see downthread) requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
- Tiberium
I've tested myself and the results are legit, this is incredible, even if a little risky (it relies on exploiting firmware): https://github.com/Zagitta/gpu-unlock/blob/main/docs/How%20A...
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
Repo for RTX 4090 and 4070 Ti Super: https://github.com/JustAResearcher/gpu-unlock-ada
- Scene_Cast2
Really interesting.
On the other hand, this only applies to a limited set of workloads (training is often memory bandwidth bound for example).
I also wonder how this affects power consumption.
- gawacer
cool