Hacker News Reader

Comment by SeanLuke | Hacker News Reader

In Java there's just Math.random(), and it's been broken, with WONTFIX, since 1998.

Java's random number generator returns exceptionally non-random values, but Sun, er, Oracle, won't fix it because of the most insane of reasons: unlike in any reasonable language, Java's PRNG essentially has a contract to be deterministic. There's seemingly a worry that someone, somewhere, is actually relying on java.util.Random to always produce the same random number sequence for a given seed from Java version to version.

View on HN

Not java but a company I worked for had a perl application where old sales records were encrypted then put on dvd's for long term cold storage. I was a lowly tape monkey at the time so the actual details of the application were far above my pay grade. but long nights between shuffling tapes and swapping dvd I would try and figure out how the system worked. One thing I found out is that they were generating the encryption keys by combining a couple of values(something like secret + date I think) then hashing. They would then regenerate the key if the data ever needed to be retrieved. On it's own this is... fine I guess. cold records in a secure(ish) facility. The worrisome part was that they were using perl's rand(seed) as a hash function. Young me was like "well I don't know much about perl but I don't think there is any documentation guarantee about how exactly the rand(seed) function works." a few quick tests later and I found out that yes rand(seed) will return different values from different operating systems. and sometime even changing the version of perl was enough. They ended up having to make sure they went back to the same system the key was generated on then regenerate and store all keys used.

The lesson I learned, don't use random() when you want hash(). random is for non-deterministic output, hash is for deterministic output. random(seed) is an artifact of implementation and should never be used for deterministic output.

The real wtf was that this was perl on win98, probably due to when it was implemented, they wanted dvd burning capability and someone sort of knew perl.

Why is it unreasonable that given the same seed the same sequence will be generated? It's useful for generative art and video games at least. Sure it shouldn't be used for security...

Because it permanently prevents all bugfixes for a major library class. And java.util.Random has some whopping bugs.

If you need determinism, then the proper thing to do is use your own RNG. There's been a good Mersenne Twister implementation on Java since 1999. But a system-wide generator should never be beholden to determinism.

If you need determinism, and the standard library is deterministic, it seems reasonable to use the standard library.

Even if its high bits go 1 0 1 0 1 0 1 0 ... ?

I would say that's a bug in the RNG, nothing to do with any determinism issue

The point is that if you document your rng as deterministic, you can't fix any bugs.

why do you believe that? Can you explain why not?

because those bugfixes will change the random numbers generated

That's fine (if you document it). Repeatability may mean 'repeatability over the same program instance/minor version/major instance'. It's fine if you're clear. But ISWYM too.

Links to these whopping bugs pls?

I also don't think you've justified why determinism from a seed is a bad thing.

Name another language where random() is deterministic. Ask yourself why that is.

As to bugs. Let's start with the famous one:

http://alife.co.uk/nonrandom/

This is due to boneheaded mistakes in Sun's choice of constants for its LCG and errors in its bit-handling. These are massive mistakes, which it can no longer fix.

Next, there's an outstanding bug in nextBytes(), which generates ints and then cuts them into bytes (a big no-no for this particular LCG).

Some unfortunate omissions: nextChar(), nextShort(), nextByte() are missing, and there is no save-state procedure. There is no nextDouble() method that is inclusive for one or exclusive for zero.

There was a notorious bug in nextGaussian() which would take the log of 0 and then divide it by 0, but that has long been fixed. :-) [And some other bugs which were fixed early on despite Sun's claim that it couldn't fix bugs due to its stupid nondeterminism promise. For an RNG!]

> Name another language where random() is deterministic

pretty well all of them, surely? eg. https://learn.microsoft.com/en-us/sql/t-sql/functions/rand-t... "For a specified seed value, the result returned is always the same."

Pretty well every language I'm aware of does it this way. Are we even talking about the same thing?

As for the others you point out, I'm afraid I can't speak for that. I'll just have to accept what you say.

Julia, for example promises that seeded random numbers will only be the same within the minor version and there's a 3rd party package if you need reproducible random numbers. Guaranteeing cross version random number compatibility comes at a pretty major cost for performance (as new better algorithms are discovered) and removes your ability to bugfix the random number generator.

Any PRNG will return the same sequence when starting with the same seed. That's basically the point of it and it's quite a useful and important property, e.g. for simulations where you then just have to document your chosen PRNG and the seed for others to replicate your results instead of also shipping gibibytes of random numbers.

And since the exact algorithm has been documented for java.util.Random you can't just change it.

That being said, .NET had a similar problem, with System.Random having some drawbacks and bugs over the years. They chose to keep the algorithm the same when a seed is used (again, it's an important property you don't just break), but otherwise switch to something completely different. It gets more complicated even, as you can derive from Random there and some of those changes may be observable, so they also check for that: https://source.dot.net/#System.Private.CoreLib/src/libraries...

Deterministic random number generation, where the same seed produces the same sequence of random numbers, can be achieved in various programming languages through different methods or libraries. Here are some programming languages and methods that allow for deterministic random number generation:

1. Python: The `random` module can be seeded to produce deterministic random sequences. 2. Java: Java's `Random` class can be seeded to produce deterministic random sequences. 3. C++: The C++ Standard Library provides functions like `srand` and `rand` that can be used for deterministic random number generation when seeded. 4. C#: C# offers the `System.Random` class, which can be seeded for deterministic randomness. 5. JavaScript: In browsers and Node.js, the `Math.random` function can be overridden to make it deterministic. 6. MATLAB: MATLAB's `rand` function can be made deterministic by setting the seed with `rng`. 7. Ruby: Ruby's `Kernel.rand` method can be seeded for deterministic random numbers. 8. Julia: Julia provides a `Random.seed!` function to set the seed for deterministic randomness. 9. R: R has functions like `set.seed` to control the randomness and make it deterministic. 10. Swift: In Swift, you can use the `arc4random_uniform` function with a fixed seed for deterministic random numbers.

These are just a few examples, and many other programming languages and libraries offer similar functionality for deterministic random number generation when a specific seed value is used.

[Sigh] Sounds are misunderstanding what I meant by determinism, my apologies. Of course a given instance of an RNG will generate the same sequence given the same seed. That's not the issue.

The issue is Java historically has guaranteed that, for all future versions and implementations of Java, the RNG will produce the exact same sequence given the same seed. It is deterministic across language versions and implementations. This is extremely unusual for a programming language, and a very bad idea. Java's RNG has grievous errors: but these bugs cannot be fixed because it would change the sequence! Its bugs are fixed in stone.

Because random(seed) has no specification for what the output will be. random() should only be used if you specifically want a non-deterministic value. The function you want to use for uniform distribution in a deterministic manner is hash(value)

I think this is no longer true:

https://www.baeldung.com/java-17-random-number-generators

https://openjdk.org/jeps/356

I guess sort of with the new proposal.

When they changed the iteration order of HashMap keys (1.7 I think?), tons of code broke, although fortunately mostly tests. It was easy to depend on it by doing something like creating a gold file test with expected results.

They don't want code to break like that, which means they need to preserve this sort of thing.

Isn't it normal to set a fixed seed in testing and then expect deterministic results so you can write simpler assertions?

Maybe its Minecraft?

Reply on news.ycombinator.com