I will be a bit sloppy with my description here: There are two ways to "prove" thermodinamics. You can start from the axioms of probability and statistics *together* with an assumption that matter is made out of atoms or similar constituents, and you can derive, through statistical physics, most of thermodynamics. That is a perfectly reasonable and practical way to do it.

However, it is extremely valuable to also understand that the vast majority of thermodynamics (including our understanding of entropy) can be derived from completely independent axioms that have no relationship to information theory or the knowledge of discreteness of matter. You need to define temperature as an transitive observable (zeroth law of thermodynamics), you need to define heat as a type of energy (first law of thermodynamics), and you need some version of the second law (there are a few equivalent ones, but the easiest is that "heat flows only from high temperature to low temperature"). Lastly, you need equations of state as a starting point (instead of assumptions about the microscoping constituents of matter). Tadaaa, you have all that is necessary to derive all our knowledge of thermodynamics without knowing statistics or atoms.

Aesthetically this is incredibly pleasing and intellectually it is quite surprising that these two sets of axioms that look nothing alike produce the same self-consistent physics. To be productive in cutting edge research you usually need to be comfortable with both.

Edit: corrected associative -> transitive as pointed out below.