The Shannon Limit (2010)
news.mit.edu
news.mit.edu
[1] http://people.math.harvard.edu/~ctm/home/text/others/shannon...
[1] https://journals.aps.org/prd/abstract/10.1103/PhysRevD.23.35...
It's surprisingly difficult to motivate complex- (rather than real-) valued quantum mechanics. See eg https://www.scottaaronson.com/blog/?p=4021, for example.
(As they point out, though, motivating "not quaternions" is pretty easy at least.)
Eh, this can be said about a lot of foundational disciplines/studies/theories. Philosophy of science (or philosophy of mathematics), logic, category theory, model theory etc would all fit the description. I'm not saying you're wrong, it's just that information theory isn't unique in this respect.
Even the distinction between that most vaunted science, physics, and it's far more fluid, subjective sibling, metaphysics is squarely settled in the philosopher's forum rather than the scientist's academy. To the scientist, the epistemic question is settled by your choice of field by necessity.
Another way to think of it: The Art to good Philosophy is to learn to cope with and identify those patterns of reasoning that degenerate down to navel gazing.
It is why the very tool by which we interlocute and communicate is in and of itself a matter of great philosophical concern. The ability to elicit a shared understanding of a purportedly common world phenomema is still hotly debated within philosophical circles. The question of what the essence of meaning is and what it means to be all happen far outside the borders of any science's capability to operate or even exist as standalone rational framework.
A science is a rational workshop, which is what is because of the collection tools and products of philosophically established ground states that render the space conducive to ongoing thought and reasoning.
Seriously. If yoy think philosophy is all deductive, crack open some books. Enrich thyself, and marvel.
Being able say and practice things like "now let's consider what this might look like in the frequency domain..." can open up radical new approaches to solving problems. "Ohhhh that signal/channel has a hard roll-off starting at 20khz... I probably need to increase/decrease my sample rate accordingly" vs flying blind with traditional time domain analysis. In practical terms, frequency domain work is mandatory for things like high-efficiency video or audio codecs.
Understanding what entropy vs information really means is a huge part of being able to competently build cryptographic primitives from first principles.
I found the following Youtube video to be better however: https://simons.berkeley.edu/events/theoretically-speaking-ma...
As a ~1-hour talk, Dr. Wootters is able to dig more deeply into Reed Solomon codes (a very popular error-correction code for nearly 50 years), as well as applications into strange stuff: like using RS codes in "Test Pooling" to save money on Syphilis tests (and probably the same methodology being used for "Test Pooling" in today's COVID19 world).
Dr. Wootters keeps things relatively dumbed down, never getting too into the weeds of the math (and indeed: only sticks with the GF(5) field, a prime field instead of talking about the more applicable extension fields). Still, extension fields follow mostly the same concepts, and GF(5) is sufficient to cover all the concepts.
Ironically, LDPC was actually developed in the 60s by Robert Gallagher, but were reinvented in the 90s when the decoding was practical.
Hamming codes are very tiny, but efficient. There are bigger (and therefore better) codes today, but hamming codes are still used when small codes are useful (or: error correcting ram)
Hamming codes are optimal and efficient for their size. But there is a need for larger codes in most practical situations. If you need single error correcting double error detecting however, Hamming is probably your best bet.