Hacker News Reader

Comment by codeflo | Hacker News Reader

These random/urandom blocking/nonblocking discussions have been going on for at least a decade at this point, maybe even longer. I get that RNG algorithms are hard. What’s surprising to me is that it seems so hard to even figure out the desired interface of such a basic thing in the kernel. Security in entropy-starved environments seems to be very hard to square with backwards compatibility.

View on HN

It's not hard by itself. These interfaces are actually pretty straightforward to design and implement; Linux made things hard for itself by designing a weirdly complicated interface with an entropy credit system, an idea that's more or less discredited now.

What's hard is accurately surveying the last 15 years of systems built on top of Linux's needlessly-complicated LRNG and making changes that don't violate the expectations of those systems. Nobody should be running "jitter entropy daemons" in userland (userland RNGs are essentially the source of all practical CSPRNG vulnerabilities in the past 2 decades), but people do, and if those systems exceeded some threshold of security before Donenfeld's patches, you can't merge those patches if they push security below that threshold.

It's a really irritating problem.

an entropy credit system, an idea that's more or less discredited now

... no pun intended, I'm sure.

Nope, just coming off a migraine and not catching stuff like that. :)

Also,the problems seems mostly to be around Linux running in a VM rather than on actual hardware. Would it be possible to have the host urandom show up as a hw random number generator to the VM and then use that for initiating things? Seems less hacky than having to fiddle with seeds. Dunno how possible that is though.

https://wiki.qemu.org/Features/VirtIORNG

Yes, and that is commonly used.

But Linux must continue to work even on VMs without that, and that is the hard part.

You can do that where for instance the VM has PCI and can plug in a virtio RNG device. But a lot of the QEMU setups Guenter Roeck reported issues with are emulating real (usually old) hardware which didn't have a PCI bus, or any other hardware RNG interface.

Also lighter systems such as routers (E.G. many OpenWRT targets)

The VMs described in the article are (faithfully) emulating real hardware lacking unpredictable timers or other sources of entropy. It’s not exclusively a VM problem; it’s a weird/old hardware problem, too.

> userland RNGs are essentially the source of all practical CSPRNG vulnerabilities in the past 2 decades

Is your point that CSPRNGs should not be implemented in userland, but instead should just read from /dev/random or do the equivalent on non-linux opperating systems?

Yes, that’s been Thomas’s position for many years[1]. And I mostly agree with him on that.

[1]: https://sockpuppet.org/blog/2014/02/25/safely-generate-rando...

> Nobody should be running "jitter entropy daemons" in userland (userland RNGs are essentially the source of all practical CSPRNG vulnerabilities in the past 2 decades), but people do, and if those systems exceeded some threshold of security before Donenfeld's patches, you can't merge those patches if they push security below that threshold.

Why not break their code? Seems justified in this case.

For better or worse Linux doesn't break userland.

> What's hard is accurately surveying the last 15 years of systems built on top of Linux's needlessly-complicated LRNG and making changes that don't violate the expectations of those systems.

Sounds like they need a compatibility or feature (cmdline) flag.

Add the flag and you can use the new improved system. Establish dates for when the old method will not be the default anymore (but can be used with the flag) and when it will be deprecated for good

That won‘t align with the never break the user mantra. And honestly I hate these timebomb solutions some API vendors put up. In a never breaking system you can‘t remove anything only add things or change stuff internally to make it work like before. They never said to get rid of /dev/random or /dev/urandom just that both would practically be the same. From reading about the problem and the different usecases and expectations it looks like it will stay like this for another decade or more.

I ran into it the first time in 2005 and it was a widely known issue, easy to find if you could find where your program was pseudo-randomly blocking (pun intended). FreeBSD already had unified random/urandom by that point.

I’m not sure if the Linux folks find something wrong with existing, time tested approaches or if there’s some NIH going on, but it doesn’t seem like a multi-decade problem.

Pedantic question: would it be pseudo-randomly blocking or randomly blocking?

It would be somewhat deterministic and predictable and therefore pseudo-random.

I don't think it's hard to figure out the desired interface. As you said, the hard part is changing the interface to what's desired decades later.

You'd be amazed at how much stupidity lingers about this issue. I don't know if the present-day kernel devs are affected by it, but an awful lot of users are.

Why is random so hard? If we assume best intentions then:

    uint64_t lemur64(void) {
      static uint128_t s =
        (uint128_t)426679527491843471ull << 64 | 2131259787901769494ull;
      return (s *= 15750249268501108917ull) >> 64;
    }

Passes bigcrush and PractRand and it won't ever cause your system to hang on boot.

What kind of wizardry is this doing?! :)

Seems that's one of these: https://en.wikipedia.org/wiki/Lehmer_random_number_generator

It's one of the simplest ways to make random numbers. Multiply state by a constant, return the upper bits/digits of state.

There's some math that goes into picking a good constant, but honestly that's optional. Usually you'd also add a constant each round but this is going for absolute simplicity.

And the line with 'static' is just an awkward way of initializing the state integer.

Those numbers are not random.

> There's some math that goes into picking a good constant, but honestly that's optional.

It isn’t optional. For example, if your constant is even, every iteration adds another zero to the end of your state, and after (in this case) 127 iterations, you’ll start returning zeroes forever.

Also, if your multiplier is smallish, the lower order ones of the current value can be used to (somewhat) predict the higher order ones of the next value.

Yes it has to be odd and it has to be the size asked for. I don't count checking that as math.

If you're particularly worried about the top couple digits of the number being undersized you could return fewer bits or use a wider multiplier.

It’s a multiplicative congruential generator (MCG), a subset of linear congruential generators (LCG). Basically a multipy (and for LCG, an add) between each generated number. Very fast, small state requirements.

Wholly unsuitable for cryptographic random numbers.

Yeah, but it always returns the same pseudo random sequence of numbers. THAT'S part of what they're trying to fix by including entropy.

what if you add timestamp?

From what source? That’s the hard part.

Think of it in terms of VM snapshots. If you boot from a snapshot, there’s no way to get any timestamp guaranteed to be secure.

But “secure” in this context is misleading. I subscribe to the “just use urandom” school of thought. It’s sufficiently secure in practice that no real security threats have emerged (can anyone point to a CVE?) and it seems like a waste of time to focus on this rather than the hundreds of other cases that cause dozens of CVEs.

The source shouldn't be an issue even if the timestamp is reused. Timestamps just don't add a ton of entropy - a timestamp is generally 64bits, so even if it were actually random you could only ever get 64bits of entropy. And it's obviously way way way less than that since the clock is probably somewhat accurate, the computer is probably less than ~10 years old, and it's probably not way in the future.

So it should be fine to add in, but I don't know how many bits you'd want to count that as.

Insufficient entropy is the root cause of this:

https://factorable.net/weakkeys12.extended.pdf

which is still happening:

https://www.quintessencelabs.com/wp-content/uploads/2020/03/...

That would also make it predictable. Essentially, if you get a sequence of random numbers, it should be quite hard to predict what the next one would be.

Yep, that is a good idea! But a timestamp only adds a little bit of entropy, and you want a few hundred if not a few thousand bits.

Then if I know about when you turned your computer on, I can guess every random number you'll ever generate.

Not all computers have a RTC.

The presence or absence of a real-time clock doesn't significantly change the problem here.

If they used a timestamp then all you really need is a couple random numbers to find the seed. Just keep guessing timestamps until you find a sequence that matches.

Here's a fun article about hacking online poker, in part by taking advantage of a time-based seed:

https://www.developer.com/guides/how-we-learned-to-cheat-at-...

"The system clock seed gave us an idea that reduced the number of possible shuffles even further. By synchronizing our program with the system clock on the server generating the pseudo-random number, we are able to reduce the number of possible combinations down to a number on the order of 200,000 possibilities. After that move, the system is ours, since searching through this tiny set of shuffles is trivial and can be done on a PC in real time."

What it will also do is to allow anyone to predict your pseudo random numbers with high accuracy. (which is bad, as the use case here is for security)

Reply on news.ycombinator.com