Comment by Panzerschrek | Hacker News Reader

Comment by Panzerschrek | Hacker News Reader

In some cases I use binary fixed-point numbers. In certain aspects they are much better than floats - no precision loss happens in addition/subtraction (if no overflow/underflow takes place), additions and subtractions are typically faster (since it's just an integer operation internally), casting from and to integers is also cheap (requires only bit-shift).

Multiplications are a little bit tricky. Multiplication by an integer is trivial. Multiplication of two fixed point numbers produces the result with the number of fractional binary digits equal to sum of the number of fractional digits in source numbers. The result may be stored in an extended type, truncated down or rounded.

Divisions work fine too, but sometimes may be slower compared to float types, because CPUs can for some reason do much faster floating-point divisions compared to integer divisions.

The only disadvantage of fixed-point numbers is that it's required to keep a balance between range and precision carefully. One can't just use some specific precision in the entire codebase, typically precision should be selected for each individual operation.

A neat trick many people aren't aware of is that you can treat binary floats as saturating fixed point, subject to some qualifications (generally the next larger float type can represent any given fixed). Float operations internally are "just" fixed point ops with some normalization steps and rounding bits on each side, so if we use a float type with enough mantissa bits to hold the fixed point value all we have to do is mask off the extra precision to get back to fixed point. This similarity to fixed point is exploited in some modern NPU hardware by storing only one exponent for an entire block of floats, with a wide fixed point unit doing the actual work, a.k.a block floating point.

This hack has some interesting advantages. Float to integer is still only a few cycles, the masking is one line of libm functions, you get better (and dynamically selectable!) precision, it has gradual underflow and overflow, you can write numeric code like usual, and normalization is automatic.

Worth noting the gap between floating point vs integer division isn't that bad on newer CPUs these days. On Zen5, for instance, DIVSD has a latency of 13 cycles vs 16 cycles for DIV.

>Divisions work fine too, but sometimes may be slower compared to float types, because CPUs can for some reason do much faster floating-point divisions compared to integer divisions.

The mantissa of a floating point number has less bits than the integer type of the same byte size.

Fixed-point can also be much more efficient in terms of bits if you know you are staying within some range. If you're working with 32 bits this can be a pretty big difference (4 billion values vs 8 million for single-precision floats).

Reply on news.ycombinator.com