Hacker News Reader

TLDR: Python gevent compiled with -Ofast messes up x87 floating point unit state. Bad for PyTorch.

I thought that the effect of these compiler flags was widely known in numerical computing. It allows e.g., reordering of floating point computations and in general disregards IEEE 754. As such, these results are expected, I’d think.

Widely know amongst very niche groups, most of whom have either been burnt by the issue or heard about someone who has and have it ingrained in their mind out of fear of debugging such a thing.

I’d bet the majority of ML people are unaware, including those doing lower level stuff.

The unexpected thing in that particular case is that even if you were well aware and avoided the flag when building your numeric code, the way some other non-numeric-computing person compiled some unrelated non-numeric module like "gevent" could result in the fast-math behaviour being applied to your code too. (Happily gcc has now fixed this.)

Reply on news.ycombinator.com