Hacker News Reader

Comment by lou1306 | Hacker News Reader

> Everything should be at least double precision by default, and extended precision serves as an additional safeguard

Honestly I don't understand how this would constitute a "solution" or even a "safeguard", really. Using any kind of FP arithmetics without being acutely aware of its quirks is going to cause headache no matter the precision. Conversely, when you do know how to deal with FP, then FP32 can suit many scenarios just fine.

View on HN

I recently tried to implement a 16bit 64k point fft in C using FP32. It thought it would work easily. Turns out it was very difficult, what worked fine in double gave huge inaccuracies in float. It was due to the large small issue, if you subtract a small FP32 from a large tje result cannot be represented so it's as if the subtraction did not happen. These kinds of errors accumulated and affected the result. I found a workaround for my particular algorithm, but it was eye opening to me that FP32 was not enough to just work in this case.

Catastrophic loss of precision is, as the name implies, catastrophic in terms of the calculation context. For scientific/engineering codes, and things requiring a preservation of resolution for proper functioning, FP32 is rarely sufficient. FP64 is usually better. For ML/AI apps, resolution isn't nearly as important.

Absolutely. If anything ML is migrating to lower precision types, like fp16, mx9, mx6, int8, etc.

Reply on news.ycombinator.com