Hacker News Reader

Comment by tomjen3 | Hacker News Reader

That isn't strange though -- if I were creating a file that I knew would only be used for speed testing, I would make it al nulls too.

It is just much easier to find a file that is some exact distance if you create it first.

View on HN

I'd use random data to make it hard to compress. You don't know if there won't be some link layer compression along the way.

That doesn't really fix the problem, what if that random data happened to compress well? The solution is to not compress anything while downloading.

"Random data' [1] will not compress well [2].

[1] high-entropy in any of the usually-available ways

[2] for any compression system actually deployed, as opposed to one contrived to match specifically the 'random' source

A large quantity of random data is incredibly unlikely to compress well.

$ dd if=/dev/random of=nonbullshittestfile bs=1M count=500

That would take forever, at least on my system, as I don't really have a lot of entropy available. Just for kicks I have started this when I began writing this reply. And now I have the huge amount of 200 Byte in this test file. So thats about 10 bytes per second.

But if you take /dev/urandom you can get quite a bit more. On this system it is about 7Mbyte/s.

What's the difference between /dev/random and /dev/urandom? I've only ever used urandom and it spits out data as fast as I can consume it.

/dev/random uses environmental noise for entropy, and can be depleted rather quickly.

/dev/urandom supplements its entropy with a PRNG so that it never blocks..

/dev/random will block while waiting to collect entropy from the system. /dev/urandom will be satisfied with pseudorandom numbers. That's fine for many applications (e.g. a file filled with garbage) but not acceptable for things like cryptography.

That entropy, by the way, is derived from the keyboard and mouse devices. If you want /dev/random to go faster, move your mouse and type more ..

Can it be configured to use other sources?

Does this mean that if I've got a server with no mouse/keyboard attached, /dev/random will block forever?

Logging into my slicehost server, and running cat /dev/random | hexdump -C seems to support this, more or less - only a few lines get output unless I start typing into the terminal - then it goes marginally faster.

You can buy hardware random number generators.

http://www.entropykey.co.uk/

http://www.idquantique.com/true-random-number-generator/prod...

http://www.gamesbyemail.com/dicegenerator

...among many others.

It will also use other interrupt timings in the creation of the entropy, most notably hard drives since they're rather random on when they reply back due to it having to rotate to the right place. Not sure how this works on an SSD. I also believe if you've got some kind of hardware RNG it will use that too.

Interesting. I don't see /dev/random blocking on Mac OS X (13" MacBook Pro). I wonder what the source is?

On OS X, /dev/random and /dev/urandom are the same thing (both acting like the traditional /dev/urandom).

I'm guessing it may be even smaller than a file filled with zeroes. If I were doing it, I'd use a sparse file, so it only takes a few blocks, but (sans compression) would take a long time to download. Like this:

  tsuraan@macbeth ~ $ dd if=/dev/zero of=abigfile bs=1M count=1 seek=1000000
  1+0 records in
  1+0 records out
  1048576 bytes (1.0 MB) copied, 0.00320282 s, 327 MB/s
  tsuraan@macbeth ~ $ ls -lh abigfile 
  -rw-r--r-- 1 tsuraan tsuraan 977G Sep 16 11:18 abigfile

I probably wouldn't actually make it ~1TB, but that's the idea.

Surely you wouldn't get as greedy as they did though? Nobody would've suspected them if they had've been a little less "cheaty".

Reply on news.ycombinator.com