I think that calories in calories out is a reasonable heuristic, but biology is more complicated than physics.
1. Different bodies have different abilities to digest different things. For example, you could eat every bit as much grass as a cow that's gaining weight, and you would lose weight (and eventually starve to death). Obviously the variation within humans is less than that between humans and cows, but it's silly to pretend that we all share identical gut biomes and identical abilities to process foods with the same efficiencies. From this, it clearly follows that some people are going to get more calories out of the same food than others.
2. Even ignoring inclination to exercise, people have different resting heart rates and body temperatures. It obviously costs more calories to heat your body to a higher temperature and for your heart to beat faster, so some people are naturally going to gain or lose weight at a caloric intake that keeps other people at the same weight.
You can argue that these differences are minor, at the margins, and unlikely to mean the difference between equilibrium and weight change, and you might be right, but it's silly to pretend that they're physically impossible because of thermodynamics. The human stomach isn't a bomb calorimeter, which is why you can't drink a liter of gasoline and skip eating for a few days.