Claude Shannon’s research laid foundations for modern communications (2020)
quantamagazine.org
quantamagazine.org
Currently our computers operate by filling the transistors with charge and/or dumping it to ground. Who even cares about in information-theoretic efficiency in that case? The cost of the actual work done is dwarfed by the ancillary cost of running the machine.
If we ever move on to something less brute-force, like reversible quantum cellular automata, I think we’ll see him as an invaluable part in the chain of formalizing what information and computation mean physically.
Kelvin/Maxwell -> Shannon -> Landauer -> maybe Bennett
My greatest concern was what to call it. I thought of calling it 'information,' but the word was overly used, so I decided to call it 'uncertainty.' When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.'
https://mathoverflow.net/questions/403036/john-von-neumanns-...
Especially as an OG (first author to use the word?): https://people.math.harvard.edu/~ctm/home/text/others/shanno...
> If the base 2 is used the resulting units may be called binary digits, or more briefly bits, a word suggested by J. W. Tukey.
[Edit: also, consider the section "3. THE SERIES OF APPROXIMATIONS TO ENGLISH" to be an early exploration of MLMs: Microscopic Language Models]
https://en.wikipedia.org/wiki/Fast_Fourier_transform#History
p.s. I started reading Tukey's textbook Exploratory Data Analysis just yesterday. It's wonderful so far, a pleasure to read. "It is important to understand what you CAN DO before you learn to measure how WELL you seem to have DONE it." "The greatest value of a picture is when it forces us to notice what we never expected to see."
A nice idea but not really.
At least the way I read Bernoulli's paper, it was more of a general musing on how to reduce risk and make insurance decisions.
Claude Shannon Invented the Future - https://news.ycombinator.com/item?id=25507627 - Dec 2020 (68 comments)