Hacker News Reader

Comment by titzer | Hacker News Reader

Do not use C/C++ for numerical code where accuracy is needed. These languages are not specified to conform to IEEE 754 and you are absolutely asking for trouble.

View on HN

My copy of C11 says

   Annex F
   (normative)

   IEC 60559 floating-point arithmetic

   F.1 Introduction
   This annex specifies C language support for the IEC
   60559 floating-point standard [...] previously designated
   ANSI/IEEE 754-1985.

C "supports" whatever the hardware and compiler feel like supporting. It absolutely does not mandate anything. In particular, there are a number of particularly simple compiler optimizations that are not forbidden, though they are not technically correct according to IEEE 754, such as algebraic reassociation and simple commutativity. Moreover, C allows subexpressions to be computed in higher precision (e.g. 80 bit "long double"), which is observable. That last one is primarily due to the x87 FPU coprocessor design that has given us a good 35 years of headaches. Good riddance to that!

> Support for Annex F (IEEE-754 / IEC 559) of C99/C11

> The Clang compiler does not support IEC 559 math functionality. Clang also does not control and honor the definition of __STDC_IEC_559__ macro. Under specific options such as -Ofast and -ffast-math, the compiler will enable a range of optimizations that provide faster mathematical operations that may not conform to the IEEE-754 specifications. The macro __STDC_IEC_559__ value may be defined but ignored when these faster optimizations are enabled.

If you make use of Clang, you won't find support for Annex F, and in point of fact you can't even macro check to see if it even will support Annex F.

The story with GCC is... More complicated. It guarantees it'll follow Annex F for only some operations [0].

So the upshot is... Most people can't tell when and how Annex F might be followed.

[0] https://gcc.gnu.org/onlinedocs/gcc-7.4.0/gcc/Floating-point-...

Keep in mind that C++ doesn't exist. The compilers all have ways of enforcing compliance.

Intel C++ seems to be pretty naughty in that it effectively has fast-math on by default.

In practice, all of the C compilers will (or can be made to) conform to IEEE 754, although #pragma STDC FENV_ACCESS ON support is very spotty (and consequently non-default rounding-mode support), although Clang/LLVM has been working on this for the past several months.

(That reminds me, I need to harangue the llvm-libc folks to add that pragma to their fpenv code for correctness sake).

No, they consistently perform optimizations that violate IEEE 754 rules, such as reassociation and commutation of expressions. These are difficult to observe (unless you are in the habit of observing -0 vs 0), but yes, you can observe them. They are very precisely specified by IEEE 754 and C/C++ compilers perform illegal optimizations.

They do those optimizations when you enable fast-math flags. Some compilers (e.g., icc) enable those by default. When fast-math is not enabled, then the IEEE 754-violating optimizations are disabled.

Source: I've been working on such optimizations this past week. And that means establishing precisely which flags I need to have enabled in order for optimization to kick in.

This is very true. Once upon a time -o3 would enable fast-math. We're very careful to ensure we use test models to verify the outputs of release binaries. Enabling fast-math as you have said optimises the equations in a way that produces different results.

The clang docs explicitly say that they don't support Annex F at all. Is this story finally going to change? So that you have at least partial support?

Op here. There is a huge amount of scientific code written in C++. Just have a look through https://www.coin-or.org/

This code was specifically C++ because we had the intention of integrating with libraries like CppAD and ADOL-C.

So, what should be used?

COBOL

Seriously? Anything but floating point if you care about precision and accuracy.

The correct - but useless - answer is: any language that's IEC 60559:xxxx conformant.

Examples are: Fortran, R, C#, Java, D and many others.

You'd need to refer to a language's specification to find out, though some aren't very well defined. zig comes to mind; it supports the data types and offers strict mode, but doesn't explicitly state its IEE 754:2008 compliance.

Last I checked C# allowed the compiler to perform the computation with higher precision than what the code requires an then narrow down the the expected precision whenever it wants (you can trigger a narrowing by explicitly casting from a floating point type to itself). Which makes writing correct floating point code practically impossible.

Reply on news.ycombinator.com