Check out the first chapter of Jayne's probability theory. It's the clearest take down of the frequentist interpretations that you have and introduction to the Bayesian interpretation.
http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...
http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...
Chapter 5 is also a doozy in terms of explaining some of the things going on currently.