Hacker News Reader

Comment by zoky | Hacker News Reader

I guess what I’m saying is that they should. We all grew up learning math before programming, and mathematically speaking, there is no qualitative difference between 15 and 14.7, and no difference whatsoever between 15 and 15.0. So why do programming languages, particularly ones like the ones you mention that purport to abstract such distinctions away, treat them differently by default? There is no logical reason why 5/2 should be 2 rather than 2.5, yet by default it is, for apparently no other reason than that is faster for the CPU to calculate.

I’m not saying get rid of these distinctions entirely. But where they matter, for performance reasons or whatever, they should be opt-in rather than the default.

View on HN

Integers are exact and give repeatable results.

Many floats (such as 14.7) are not even represented exactly, float operations have a large number of issues.

If you're wondering about all this, see:

What Every Computer Scientist Should Know About Floating-Point Arithmetic

~ David Goldberg, March, 1991 issue of Computing Surveys

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...

(and elsewhere)

It's a bit like musical scales. It all seems nice and neat, until you start looking for exactness.

Please don’t spread FUD about floating point arithmetic. IEEE754 is an extremely carefully worked out standard which provides a good trade off between speed and accuracy. The problem of representing a continuum of real numbers in a fixed amount of memory is highly nontrivial. The error incurred by each operation is bounded and predictable—not exact, but as good as you’re going to get given the constraints. Also, results are replicable. Speaking as someone working in numerical analysis and scientific computing, so I work with floats all day long.

> Please don’t spread FUD about floating point arithmetic.

I'm sorry, my apologies.

Are you saying that IEEE754 does exactly represent 14.7 ?

> Speaking as someone working in numerical analysis and scientific computing

You're hardly the only Scot in Glasgow.

I feel certain a few here abouts have arc'd through PDP-11's, Cyber 205's, and landed in modern centres such as Pawsey.

No, I’m not. In the applications IEEE754 was developed to support through standardization, being able to represent 14.7 exactly is simply unimportant.

... unless I have $14.70 in my bank account, and some naif was handed the job of writing the code to track that and didn't know about the non-intuitive consequences of relying on the IEEE floating point standard. Then its limitations suddenly become very meaningful to me or my financial institution.

I guess you're right. We should throw the baby out with the bathwater because a bank hired a crappy engineer. There is no other solution to this problem.

I don't think anyone's calling for that. I think they are commenting on the fact that floating-point is an excellent tool for a special case of computing (a tool which has some significant drawbacks if the representation is treated as equivalent to another representation) and most languages we use make it frighteningly easy to slip-slide from one representation to the other without noticing when they are, in fact, extremely different representations of numbers and numerical computation.

It isn't frightening if you've been properly trained and understand how your tools work. The onus is on the bank to train its employees to use the proper number type for money.

I have no problem with languages having first-class support for big integers, arbitrary precision floating point numbers, arbitrary precision rationals, etc. The more representations the merrier, provided they have an efficient implementation.

Having programming languages where floating point types are first class, unboxed types is extremely important for anyone who writes software which crunches numbers. I would say this is more important than providing training wheels for inexperienced programmers.

It's not so much that floating point needs to be boxed as that all-too-many languages allow smooth (read "sloppy") casting from float to int and back, often with semantics that are unclear to the developer and require esoteric memorization.

In 5 seconds or less, can you say whether 1.0/4 will be 0.25 or 0 for every language you use regularly? Most developers can't, and most languages make statements like that trivial (and worse, it far more often shows up as "x/y" for x and y of types that need to be looked up to know what the evaluated type will be). There's a reason C++, for example, has the MISRA rules about avoiding implicit coercions (because even though what happens is [generally] well-defined, enough code in the wild has been broken for developers mis-remembering or mis-applying the rules that explicit casts are highly recommended for high-reliability code).

OK, I get the point you’re trying to make now.

The languages I work with most often in decreasing order of frequency are: C, Python, C++, Fortran, MATLAB, and Julia. I don’t regularly cause problems for myself by running afoul of implicit casting rules, but I also spend all day writing numerical software. I can see that it can cause problems for other people who don’t spend as much time dealing with IEEE754.

There are definitely days where I feel like an OCaml-like level of casting pedantry would be nice, and other days I feel like it would be exhausting and make my code harder to read. I’ve never written a big piece of code in a language that required explicit casts for everything. I’m open to it.

I would say that unsigned integers cause me the most problems of any numeric type, but not so many problems that I’m looking to change languages. If I worked in a different field it’d be a different story I’m sure.

By the way, I had a chance to check: and I was right with my guess. All six of those languages evaluate 1.0/4 to 0.25, exactly as expected. It might have taken me 30 seconds or so to form my answer ;-) but that time was mostly spent parsing your question.

What languages don't evaluate 1.0/4 to 0.25? I checked OCaml (exception), node (0.25), Lua (0.25), Swift (0.25), sbcl (0.25), ... and then I got bored.

To me the expected behavior should be something like: for a binop between a float and an int, promote the int to float and do the floating point operation. Obviously there are corner cases here...

> Are you saying that IEEE754 does exactly represent 14.7 ?

Pedantic: You are aware that 754 also specifies a decimal model?

The neat thing about floating point is that it works well to represent real-world values.

It turns out that the real world is amazingly inexact, to the point that the inaccuracies of floating point just never even show up.

That's exactly what the other commenters are arguing about. Why should 14.7 be not representable exactly when 15 is. With python you can use Fraction(147, 10) or Decimal("14.7"), but you have to explicitly opt-in.

But, go further. Why shouldn't 1/3 be represented exactly when 1/5 is?

If it makes you feel better, with our current system, 1/256 is represented exactly. Using relatively few bits, all told. :D

But 1/3 can be represented exactly as Fraction(1, 3).

:D That was my point, you can get there with other representations. It amuses me that Common Lisp lets me type that as 1/3. Such that (+ 1/3 1/3 1/3) gives the expected answer.

This isn't without drawback, of course.

Fractions, or the rationals are equivalent to polynomials. The rationals are also in aleph-naught, and thus have a one to one and onto mapping to the natural numbers.

Floating point numbers are equivalent to rationals with their base as the denominator I. base-2 rationals with denominators that are powers of 2 will have termination expansions.

Useful for storage but not arithmetic as Fraction(147/10) has 10 in the denominator and 5 is a prime factor that isn't in the base.

BPD, or Binary Packed Decimal also allows for the storage of 14.7

But once again not useful for arithmetic.

Accumulated errors, catastrophic cancelation, etc are real problems too.

In decimal, 1/3 can be stored exactly but the expansion is 3 33333...

Because 3 is prime and not a prime factor of 10.

Had we decided on something rather than 10 for our base it may have worked out for computers better, but it is what it is.

The rationals are nice, being the only countable, totally ordered sets. But producing a computer circuit that can perform arithmetic on them isn't practical as far as I know without using fixed radix and the associated problems with non-terminating expansions.

The probability of a real number being exactly 14.7 is zero, so it is immaterial if you can represent hat particular real exactly, as long as the basic operations on the reals work as expected. Support for == between floats arguably should be removed, as it isn’t really a valid concept.

The probability of a real number equaling 14.7 depends on the specific distribution the real number is being sampled from. If X is a Bernoulli(p) random variable, and Y = 14.7*X, then P(Y = 14.7) = p.

Support for exact equality between floating point numbers is a perfectly useful and valid concept and should *not* be removed. You can easily design a simulation such that if you run it twice, it will give exactly the same result bitwise. This can be utterly invaluable when debugging complicated numerical software.

sampled from reality i guess I was implying. Whatever model you have that generated 14.7 depends on measurements from reality with a bit of error bars.

I admit removing language support for == of floats is a bit of a wild idea, but it is conceptually sound, if not practical. I'd support keeping it in a test/debug package.

Tho if you are looking at bitwise patterns, then I find memcmp or its equivalents to be needed, to handle Nan != NaN.

NaN != NaN is very useful when debugging numerical software. NaN is a great placeholder value for things which don't have a clear default value to which they can be initialized.

To your point, there are plenty of situations where a floating point number will be initialized to a magic constant (0, 1, infinity, so many others...). Beyond initialization, there are also loads of functions which frequently yield floating point constants which can reliably be tested for equality, like max(0, x). There are no error bars here. Exact floating point equality can usefully tell us things like "has this number changed", or more generally "has this special case occurred"/"have we reached a particular state".

Removing floating point == isn't conceptually sound, and it isn't practical.

I hope you aren’t testing for equality with pi or e or rho or square root of two. One and zero and -zero might be special cases. If you are testing max(0, pi) also you will be sad sometimes. Especially tho if you are doing computation and then testing for equality you will mostly be wrong.

What are the cases you are mentioning? Surely something more complex than a float for a loop variable? When the error is less than a threshold, when the sign changes, when the number of objects 8n a partition is one, I can’t think of a calculated float that would be a test for something with equality.

> We all grew up learning math before programming, and mathematically speaking, there is no qualitative difference between 15 and 14.7

IMO, there is a huge qualitative difference between 15 and 14.7; 15 is an integer, while 14.7 isn’t.

I also think most people would know that instinctively. Reason? I think most people learn arithmetic progressing more or less along these lines:

  - positive integers
  - non-negative integers (i.e. learning and understanding the concept of ‘zero’)
  - non-negative rationals
  - all rationals
  - decimal fractions
  - all reals

Phrased another way: I would think the majority of people would learn to count to 15 long before learning what 14.7 means.

Also, as sfpotter already said, switching to doubles only isn’t a panacea. Neither are bigints (won’t help representing the result of ⅓ exactly, for example), rationals (won’t help representing the typical hypothenuse of a triangle exactly, for example), or radicals (won’t help computing the area of a unit circle exactly)

A CAS would help, but by then, you’re giving up lots and lots of performance.

(that doesn’t mean I am a fan of languages that silently round integer division. Silently moving to bigints and rationals IMO can be a good choice, but silently moving from an exact type to an inexact one IMO never is)

> IMO, there is a huge qualitative difference between 15 and 14.7; 15 is an integer, while 14.7 isn’t.

Yes and no. The fact that one is an integer and one isn’t only really matters when considering them in the context of a set, e.g. when counting. But there’s nothing special about integers that affects how we use them relative to decimals. Multiply both of them by 10, and you’re back to integers. Divide them both by 10, and you’ve got decimals. It’s perfectly reasonably for all mathematical purposes to consider 15 and 15.0 to be equivalent to each other. This isn’t the case with computers, and that has nothing to do with mathematics.

> We all grew up learning math before programming

Speak for yourself :)

Reply on news.ycombinator.com