I don't think anyone is arguing that calories don't matter, what matters is if a free living humans can eat a diet without too much effort. I don't think many people would want to (or even could) eat 5000 calories of cream cheese. But you probably could do it if you're eating ice cream and potato chips.