Comment by fdr | Hacker News Reader

Comment by fdr | Hacker News Reader

Perhaps more pervasive and despairingly problematic is number representation. JSON only, per specification, supports arbitrary precision numeric representation. However, Javascript is -- also per specification (as I understand it) -- entirely floating point. That makes working with money in Javascript somewhat hazardous. While there are various arbitrary precision libraries out there for Javascript to assuage this, the problem is most JSON-parsing routines will always force one through a floating point conversion anyway, so a loss of precision is more or less inevitable.

While swiftly coercing raw off-the-wire number representations to one's arbitrary precision library of choice can avoid most cases of noticeable accumulated error, it is irksome that the only way everyone seems to get by is by cross their fingers that any loss in precision caused by "JSON.parse" is meaningless to their application.

Or, the problem can be soundly avoided by using strings in the JSON payload, which is lame but effective and probably one's best practical choice. It is clearly an example of corruption spreading from Javascript to an otherwise reasonable feature of the data representation format JSON.

The only real solution for high precision numbers in JavaScript are string-based "big num" classes, unfortunately. Sending important numbers (banking, gas prices, etc) as the built-in number type is just setting yourself up for pain.

JavaScript uses IEEE754 doubles for numbers, so as long as you can count on all of your parsers to do that, then you can rely on any integer up to 2^54 being represented exactly. For money, work in cents.

You mean, for American money, work in cents.

The name cent shouldn't throw you off here as applying only to US currency. Many other currencies have a notional 1/100th unit of the basic one, that usually also exists as a cash value.

So s/cent/atomic subunit/, because "for money, work in cents" is meaningless for yen and ambiguous for Bitcoin ("bitcents" or "satoshis"?)

If only people reading my comment had some sort of brain with which to interpret it instead of being doomed to apply only the exact literal meaning.

I'm afraid you ask too much of the Internet.

Yes, he clearly should have taken into account any possible subunit denomination, including exotic native tribe money units like sea shells, because international programmers reading HN are not expected to think of that of their own.

Such vitriol! Can your currency code handle sea shells? It should! :)

I only mention it because the yen thing surprised me once as a newbie programmer. (A tip: "%0.2f" is no substitute for actual i18n.) It never hurts to be aware of the edge cases.

>Can your currency code handle sea shells?

I'm afraid it mine can only handle sea shells!

>I only mention it because the yen thing surprised me once as a newbie programmer.

So what's the yen thing? How does it differ?

The yen has no fractional units (anymore), so when I added support for that currency but continued to use the same format string as all the others, prices like ¥1000.00 looked unnatural.

The historical subunit of the yen is the "sen" (銭), which you often seen mentioned in prewar books. Postwar inflation made sen-denominated currency obsolete, but...

[A bit of googling also turns up the "rin" (厘), which is a thousandth of a yen...]

That's faster, but I think can be a poor trade-off because:

1. The speed gain is negligible for most programs

2. The addition of any pricing that requires fractional cents will require careful work to handle unit conversion to maintain integral representation.

#2 becomes ugly when one has an API many people use, and one must bother them to update their code paths to use the new, higher precision that can handle all money as integers. This is not even counting the case where one tries to cut a corner and someone ends up not doing the conversion they ought to.

Also, programs often are more lucid when operating in terms of the frequent units of choice, such as dollars or fractional dollars. Few domains price everything in cents by preference, because in aggregation often dollars -- sometimes many -- are exchanged. The problem gets worse if one needs weird units like 0.1c to regain integral numbers.

There are ways around these, but falling back on strings seems to me the lowest-maintenance option.

On the other hand, integral representations have few dependencies (a compliant javascript interpreter), which is also a pretty big plus for that.

Work in 1/100ths of cents? You will have to convert those strings to numbers at some point (or use a specialised library format).

The latter. Such as big.js (not an endorsement, my understanding is too weak for that).

Just about every practically used programming environment from every walk of life has such a thing in the standard library, and I suppose they are there for good reasons. However, the fact is that Javascript doesn't have one, so one will have to weigh dependencies into their decision.

Go - math/big

Java - BigDecimal

Python - decimal

Ruby - BigDecimal

PostgreSQL - numeric

Interesting mention: decimal in C#, deemed sufficient for monetary calculations at 128 bits of precision.

Another interesting mention: Oracle, which in my understanding handles just about all its numbers this way, by default. This might tell you something about their early customer base.

Lisp - rationals http://wukix.com/lisp-decimals

CS history lesson -- don't use rationals for money. You have no business messing with the denominator, so the extra freedom of using arbitrary rationals is just more rope to hang yourself with.

Incidentally, 1/100th of a cent is a "milray", defined in A Connecticut Yankee In King Arthur's Court.

It's 2^53 (9 quadrillion).

Math.pow(2, 53) === Math.pow(2, 53) + 1 // true

Math.pow(2, 53) === Math.pow(2, 53) - 1 // false

Yep, remembered "53" but then misremembered it as the number of mantissa bits.

Many domains, from gas stations to stock markets, require sub-penny pricing.

If sub-penny pricing is needed, usually myridollars (10^-4) suffice. Your example requires millidollars. I'm curious if there's a natural use case requiring sub-myri resolution.

Securitized bonds present many such cases.

The idea is that you have income for each loans, which then pay into bonds based on whatever rules may have been set. These rules often have terms like a fixed interest on the outstanding principal for senior bond issues, and then various divisions for the junior, with a weird "IO" piece that gets the leftovers that don't divide neatly. The rules can be anything that they were structured to be. The result surprisingly frequently is something where the allocation of the final penny in billions of dollars can be impacted by floating point ambiguity. (And the prospectus seldom will clarify this - the ultimate control lies in whatever the servicer's computer program does.)

Hopefully we won't see billions of dollars being handled in javascript.

Even with securitized bonds, myridollars are sufficient to represent the payments.

Do not confuse "unambiguous" with "correct". If you use myridollars and the servicer used floating point with roundoff errors, the servicer's implementation is by definition correct.

All of the interchange formats that I can think of at 1 AM (including FIX and exchange-proprietary formats) are very specific regarding the data. For example, the FIX floating point specifically limits the number of significant figures to 15

Minor electronic components like surface mount capacitors have tiny prices. I just looked up a model that costs $0.00745.

EDIT: Millage taxes on property are denoted in mills—thousandths. The taxes are often fractional mills, like 2.225. So you need 10^-8 resulution at the least.

I'm guessing those capacitors are sold by the reel, maybe 4,000 to the reel.

Yes.

The "work in cents" thing usually works for same currency, but not when you involve foreign exchange:

Up until 4-5 years ago, the EUR/USD or GBP/USD or GBP/EUR were quoted with 4 decimals; Nowadays, there are quite a few places where you have to deal with the 5th decimal to make the books balance. It's going to be a roundoff error either way, but it might be a round-off error that keeps been counters, auditors and therefore eventually IT people awake at night.

"Works in cents" can be generalized for an integer with implied decimals. 4 or 5 decimal places gives you plenty of room to work with if you are just using a double to exactly represent an integer value.

That's almost never true using 32 bit math, is increasingly less and less true for 64 bit math (e.g., if you need 5 decimal places, you're left with just 13 digits - that's only trillions!) - but most importantly, a support nightmare - a lot of software written until 2002 or so assumed 4 digits is enough - Microsoft stuff (e.g. Visual Basic, I think other stuff too) had a "money" 4-decimal-digit fixed point type.

And then the market adopts a 5th digit - and you have to essentially rewrite all the math stuff.

I'd hope you would abstract the representation so that you don't have to rewrite everything just to add another digit of precision....

Well, yes and no.

If you're using language support (e.g., in VB until 2004 "dim cost as Money" was a declaration that was good for every purpose), then you're out of luck, and you do need a rewrite.

But even if you don't - this is far from trivial: Is your own implemented money type a fixed point type? If so, what is your fixed point? (e.g. Yen only needs 3 after the dot, but trillions in the front is not enough; GBP/USD needs 5 after the dot, but trillions usually IS enough).

Is it fixed per currency? Is it floating decimal point? The abstraction here is far from trivial if care about performance.

Of course, if you've used a language or a library with a usable decimal type (I'm sure there's other, but I've only been happy with Python), the abstraction has been taken care of.

But generally, abstracting money (value+currency) is not as simple as one would assume, and it is very rare that a production system gets it both right and future proof.

>(e.g. Yen only needs 3 after the dot, but trillions in the front is not enough; GBP/USD needs 5 after the dot, but trillions usually IS enough).

At that point screw it, 64.32 for 18.9 digits and round it to 128 bits with a currency code. Comes with built-in unit checking.

I don't think any of the common business programming languages let you add new types to the numeric tower. C++ does, through operator overloading. Haskell has very nice support for this; to get started all you need is "newtype Money = Money Rational deriving (Show, Eq, Ord, Ratio, Integral, Num)". If you later want to make the type more restrictive than Rational, you can do that without changing any calling code (which will work on normal Num instances, most likely).

Unfortunately, "Num" isn't precisely what you want in money - you don't multiply dollars by dollars to get dollars...

But it works well enough.

Rather than arbitrary multiplication, you should probably have an OrderUnits type, then define a function of type Money -> OrderUnits -> Money that calculates the price of multiple units.

(Money 3 * 2 will return Money 6 if fromIntegral 2 = Money 2. But that is not really the operation you want to perform, so you might consider not implementing Num for Money.)

Right, exactly.

Lua does not provide integers, too. From their mailing list I have learnt that the IEEE rules guarantee, that e.g. doubles have absolutely now problem representing 32bit integers. You write the same here. It seems to not be intuitive, but IEEE took care of it.

But I wonder why you write that for values in 1/100 one should multiply by 100! Wouldn't an IEEE double have no problems with values with two decimal points, too? Or are there many problems with that, e.g. that $0.01 + $0.01 might already cause rounding errors which do not even out by the clever IEEE rules?

Of course 2^54 woudln't be the highest value where one shouldn't worry anymore, but something smaller.

1/5 in decimal is 0.00110011... repeating in binary, therefore not exactly representable, therefore it could lead to errors. Not big errors, but enough that smart people would prefer not to use it for money.

To give a concrete example, 19.99 can't be exactly represented in floating-point, and ends up being 19.989something. If you have a price of $19.99 that's represented in floating-point, and if you truncate rather than round as you pass it off for display, you'll end up displaying a price of $19.98. With enough manipulation, the error can accumulate to the extent that rounding the result won't save you either.

JavaScript allows all the same numeric literals in its syntax, it just imposes additional semantics on them.

If I remember correctly, this was a problem with Facebook user IDs. I believe they ended up making the value a string in the API to deal with this (and padding new user IDs by some large power of ten so that developers would hit this code-path early enough to catch it).

> However, Javascript is -- also per specification (as I understand it) -- entirely floating point. [...] While there are various arbitrary precision libraries out there for Javascript to assuage this

Interesting! I hadn't seen those before.

That strongly suggests that browsers ought to add some built-in extension types for arbitrary-precision arithmetic, using fast native libraries like GMP. That would then allow such JavaScript libraries to use the native types when available, and their existing pure-JavaScript support when not.

Browsers shouldn't do it. Ecmascript should do it.

Eventually, sure; I'd love to see some standardized signed/unsigned integers, for that matter. However, in the meantime, a browser could add an extension that existing JavaScript libraries could transparently use when available.

Reply on news.ycombinator.com