Rust Compiler Refuses to Auto-Vectorize a Simple Float Loop
Trying to Make a Loop Auto-Vectorize
A dot product over two f32 arrays compiled with Rust 1.98.0 and x86-64 yields only scalar SSE instructions, not SIMD. The culprit: IEEE 754 float addition isn't associative, so the compiler can't reorder operations to vectorize. An integer version auto-vectorizes immediately. Manually chunking the float loop into groups of four finally triggers vectorization, but limits the compiler to 128-bit registers even when AVX2 is available.
The main problem here is that the compiler can't safely reorder floating-point arithmetic operations because IEEE 754 floating-point arithmetic is not associative, i.e. (a + b) + c is not necessarily equal to a + (b + c).
- dzdt
Matt Pharr (author of the SIMD programming language ISPC) nailed it with the insight "Auto-vectorization is not a programming model."
As Matt writes ([1]) : "
The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit."
- gnufx
Vectorization doesn't imply SIMD, of course. The first vectorizing compilers were for CDC(?) systems long before SIMD. Today you have SVE in Arm, for instance, distinct from SIMD Neon.
Anyway, I'm familiar with optimizing numeric loops in C (and Fortran) rather than Rust. I've rarely seen simply using SIMD intrinsics work where GCC auto-vectorization didn't with the same semantics (like numeric equivalence in reductions). In most cases you can get away with -fassociative-math, of course, and not sacrifice peak performance, e.g. BLIS passes its extensive tests with it on, but you should check, of course. (GCC also documents the option as necessary to get Arm (Neon?) to vectorize at all.) Most of the time when people tell you how much better the Itel compiler is, it's because it incorrectly defaults to something like -funsafe-math.
Regardless, GCC (like other compilers) will tell you about vectorization with the -fopt-info- options without examining assembler, and you can have some surprises. For instance, you use unsigned in C for loop indices that you know are positive, and see failed vectorization due to "loop not affine", because of C's overflow semantics; use signed types instead.
There's another reason for using properly-optimized numerical libraries (typically BLAS), is that, at least for level three (matrix-matrix) operations. Even if you get the blocking right for the memory hierarchy, you typically won't get peak performance just with vectorization because tri […]