The Hidden Cost of Floating Point: Five Slower but Exact Alternatives
More Floating Point Alternatives
Floating point isn't the only way to represent numbers. This comic outlines five software-based alternatives: decimal floating point (base 10, standardized in IEEE 754), exact fractions, symbolic computation (e.g., sqrt(2) instead of 1.414), interval arithmetic for tracking error bars, and binary-coded decimal, still used in financial formats like ISO 8583. All are slower than hardware floats, but each solves precision problems that binary floating point can't.
These are all implemented in software (not hardware) so they’re a lot slower, and different languages have different libraries.
- AlotOfReading
Usually, if you know enough about your algorithms to select an appropriate float alternative, you also know enough to fix your float code and that's what you should actually do.
That said, some of these aren't alternatives. Symbolic computation is a different thing entirely. Interval arithmetic can be built atop floats (e.g. IEEE-1788) and has its own zoo of unintuitive behaviors. BCD is better called a historical artifact than an alternative these days.
It's really just rationals and decimal floats in this list, which probably don't solve the issues you have if you're considering float alternatives.
- ashton314
Also of interest: Herbie analyzes your math expressions and helps you figure out where FP error accumulates. https://herbie.uwplse.org/demo/
We use floats as a trade-off between speed and accuracy. IEEE 754 is a very reasonable trade-off for a wide range of applications, but if you can figure out where you need to trade speed to get more accuracy with e.g. one of the methods mentioned here, Herbie's gotcha covered.
I remember seeing some research about switching between formats, but I don't have anything to cite right now.
- Panzerschrek
In some cases I use binary fixed-point numbers. In certain aspects they are much better than floats - no precision loss happens in addition/subtraction (if no overflow/underflow takes place), additions and subtractions are typically faster (since it's just an integer operation internally), casting from and to integers is also cheap (requires only bit-shift).
Multiplications are a little bit tricky. Multiplication by an integer is trivial. Multiplication of two fixed point numbers produces the result with the number of fractional binary digits equal to sum of the number of fractional digits in source numbers. The result may be stored in an extended type, truncated down or rounded.
Divisions work fine too, but sometimes may be slower compared to float types, because CPUs can for some reason do much faster floating-point divisions compared to integer divisions.
The only disadvantage of fixed-point numbers is that it's required to keep a balance between range and precision carefully. One can't just use some specific precision in the entire codebase, typically precision should be selected for each individual operation.
- thebeardisred
Don't forget Posits (Unum). https://www.cs.cornell.edu/courses/cs6120/2019fa/blog/posits...
- timschmidt
Oh hey, one of my projects is relevant to an article: https://github.com/timschmidt/hyperreal
It's an infinite precision exact constructive real with excellent performance characteristics and approximation only at explicitly named lossy export functions.
Some recent benchmarks: https://github.com/timschmidt/hyperlattice/blob/805d092d1d96...