Subnormal Floating-Point Numbers Are 50x Slower on Intel CPUs
Subnormal floating-point numbers are expensive on Intel processors
Daniel Lemire benchmarks subnormal floating-point arithmetic across Intel, AMD, and Arm processors. On Intel Xeon, multiplying subnormal numbers is 45–50 times slower than normal numbers, and a dependent chain jumps from 1 ns to over 30 ns per step. AMD Zen 5 and Arm chips handle subnormals at near-full speed. Even 1% subnormals can slow vectorized Intel code.
On Intel processors, a multiplication involving a subnormal number is about 45 to 50 times slower than a multiplication over normal numbers.
- cesaref
It used to be quite normal to add a low level random signal to inputs when writing DSP code so as to avoid dropping into subnormal territory. Careful analysis of the algorithm would identify any points where this was also necessary (e.g. feedback paths when running delays).
Obviously those lucky/unlucky enough to be writing 56k fixed precision code wouldn't have this concern, but other ones instead :)
I think flush to zero is probably the preferred strategy these days.
- khuey
If you don't _need_ subnormals MXCSR.DAZ/FTZ (which you can get gcc to set via -mdaz-ftz) will let you ignore all of this.
- pixelpoet
This has been the case since a zillion years, since the Core 2 Duo days at minimum.
- juancn
Apparently it only happens on P-cores, recent E-cores have a fast path for subnormals.
- gwbas1c
I'm still trying to understand what a subnormal number is; IE, I'm looking for the TLDR so I know just enough to know if I'm using them and need to learn more.
Unfortunately, the Wikipedia article, while probably being accurate, doesn't give a clear and concise answer.
IE, is 0.0001 a subnormal? Or is it 0.000000000000000000001?