I can accept that. What I don't quite understand is how restricted your concept of "physics" is.
Can physics explain that if you put together a glass of hot water and a glass of cold ethanol you end up with a warm mixture of water and ethanol?
If the answer is "no" that's fine! Maybe physics cannot explain much for you.
If the answer is "yes", I wonder how can you do statistical mechanics without dealing with probabilistic concepts.