Spying on floating point behavior of existing unmodified scientific applications
blog.acolyer.org
blog.acolyer.org
When asked to divide 2 float numbers by another 2 numbers, clang compiler used division instruction for 4-wide vectors (for 32-bit floats, modern hardware only implements 4-wide and 8-wide vectors), that code runs 0.0 / 0.0 in the unused higher 2 lanes.
That code is correct (we only want 2 lower lanes, and the result there is good), but FPSpy would then show these red lights for that software.
> maytrap: The compiler avoids transformations that may raise exceptions that would not have been raised by the original code. Constant folding performed by the compiler is exempt from this option.
https://releases.llvm.org/10.0.0/tools/clang/docs/UsersManua...
You can see here that this prevents clang from using vdivps: https://godbolt.org/z/WG7bEE.
Yes. It even triggers a runtime exception when these are enabled in the corresponding bit of MXCSR status register.
> I don't think compiling code that doesn't divide by zero into code that does is correct.
There’re good arguments both for and against.
Personally, I think it’s fine. The optimization clang did almost doubled the performance. Most people don’t touch these bits in MXCSR, and the default values are good.
When manually vectorizing, I sometimes manually writing code like that when the alternatives are much slower. In this particular example it’s possible to fix for free by replacing vmovsd with vmovddup, _mm_loaddup_pd in C++. But it’s not always the case, e.g. when dealing with 3D vectors you need extra instructions.
In SSE the fastest way is _mm_insert_ps, it can either copy 3-rd lane into 4-th one, or selectively zero out lanes, a single very fast instructions that doesn't touch memory. In AVX however you would need memory access + _mm256_blend_ps.
The whole thing was a balancing act of making sure that solutions were convergent and not unstable. Floating point numbers would fall in that category. Ie its OK to be a little a tiny little bit wrong as long as the final result converges on something reasonable. Especially given the other sources of error are magnitudes higher (for instance, picking a timestep that is way too large, relaxation factors when solving... I don't remember exactly but it felt like a bit of an art within engineering to get the thing to be reasonable.
https://www.flow3d.com/resources/cfd-101/numerical-issues/nu...
I think the article says “replace the relatively rare inexact calculations by (more expensive) calls to libraries that produce correct results, then stuff the result in a IEEE float/double, so that the less rare calls can run cheaply. Problems with that approach:
- a computation at a given location in the code can be exact zillions of times, only to be inexact on the zillion+1th call, so instrumenting cannot prove a computation to be safe for doing on the hardware.
- worse, the “stuff the result in a IEEE float/double” will cause inexact rounding (there may be extremely rare instances where the value of a trig function or something equally complex is exactly representable, but the CPU doesn’t compute that exact number, but those will be extremely rare)
Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG
The "inexact" exception message you got at the end of essentially anything on SunOS at one time was quite annoying. (I can't remember whether that was specific to the SunPro compiler.) Fortran provides control over this with IEEE_ARITHMETIC, but I don't remember whether that's actually part of the standard.
Edit: a SO question about "the LD_PRELOAD trick": https://stackoverflow.com/questions/426230/what-is-the-ld-pr...