Hacker News Reader

I'm reading that post, and while fascinating, I don't understand the entropy calculation.

The digit chance is easy:

For a single digit to appear one needs 1/3 * 1/10 = 1/30.

For a single letter to appear one needs 1/3 * 1/26 = 1/78.

But the bits of entropy throw me off. 26 + 26 + 10 = 62, which is 2^5.954. But that is for a uniform distribution. The writer states that it actually is 2^5.826, or 1/~56.7. I don't get how they to that number.

View on HN

Here's how you can get that entropy using the Shannon entropy equation from https://en.wikipedia.org/wiki/Entropy_(information_theory).

Like you said, the probability of any digit appearing is 1/30, and there are 10 digits. The probability of a lower- or upper- case letter appearing is 1/78, and there are 26+26 = 52 letters.

Plugging that into the formula for Shannon entropy, we get this:

- 10 * (1/30) * log_2(1/30) - 52 * (1/78) * log_2(1/78) =~ 5.826

Reply on news.ycombinator.com