For anyone interested in these ideas, I'd highly recommend reading Shannon's original 1948 paper (linked in the article). While technical and precise it is actually quite digestible for someone with a bit of a quantitative background.
direct link to PDF: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=677...