Here is a great primer on entropy that you might find helpful - http://colah.github.io/posts/2015-09-Visual-Information/