Hacker News Reader

Comment by colanderman | Hacker News Reader

Yes, we're both talking about sin(pi) (hence why I claimed "sin(pi ± epsilon) better damn well return 0").

I take issue with this assertion:

It would be absolutely wrong to round a non-zero number to zero.

Why? I know IEEE floats have subnormals and subnormals can be VERY small, but if the error is less than half the minimum representable float, why should it be rounded away from zero?

Yes, zero is "special" because you can't divide by it, but for stability, my understanding is that you shouldn't be dividing by anything that can be so close to zero as to potentially round to it anyway.

View on HN

sin(pi+eps) is a quantity on the order of eps. Not only is it non-zero, it is also so far from zero that it can be represented quite accurately with a floating-point number (which is a separate concern from the accuracy with which it is computed).

The relative error introduced in rounding 1e-16 down to 0 is 100%, or approximately 1e16 ulp, which is why that would be unacceptable. Floating point arithmetic cares about relative errors much much more than absolute.

Also, while dividing by zero is a problem, dividing by 1e-16 should not be a problem, at all. 1e-16 is a perfectly ordinary representable number.

[eps] is also so far from zero that it can be represented quite accurately with a floating-point number

What is missing from your claim is where you say, "Specifically, the delta between pi and its float-point approximation is greater than half the minimum representable float." This is almost certainly true of pi, but it is NOT true of all real numbers! 1+(1e-1000), for example, fails this test. You cannot – logically! – make the claim that the error of any close approximations of real numbers should not round to zero because the error is not small enough; clearly in many cases it is!

Yes, dividing by 1e-16 is not a problem! My question is whether this is a desirable thing to do in an algorithm that should be stable.

Floating point arithmetic cares about relative errors much much more than absolute.

This hints more at a reason, but why is this true? Yes, I know floating-point numbers are spaced logarithmically, and yes, relative error is obviously the correct measure when dealing with multiplication and division. But if you are not dividing due to stability concerns, so your value eventually finds its way to an addition, isn't absolute error more important?

Nobody here besides you is talking about numbers that are less than half the minimum representable float.

Edit: Oh my god now I get what you are saying. There are plenty of implicit reasons why the value would end up being non-zero -- in particular, any unit-scale continuous function would have to be artificially toxic for epsilon to be smaller than the smallest float. There is no reason for the article to point out that nit.

Well, thanks at least for taking the time to understand my point. Though I don't consider it a "nit" when the article implies that "never round to zero" is some sort of rule (such as "round toward even LSBs" is), as opposed to a consequence of the properties of floats and the constants and functions involved.

The sign-exponent-mantissa nature of IEEE format means (basically by design) that you have a much finer resolution near 0 (smaller absolute value) than you do away from 0 (larger absolute value).

We're talking about sin(x) with x ~ pi. In this situation the output's absolute value is much much smaller than the input's absolute value. So the output will have much finer resolution than the input.

The output being appreciably non-zero comes from the output having much higher resolution than the input. The output wouldn't be anywhere near small enough to be subnormal.

Reply on news.ycombinator.com