I’m tracking my calories intake for more than 2 years. Every meal, every day (due to my fitness goals and I’m very serious about it) and I totally agree, that if you haven’t nazi-tracked your calories at least for a while, you have totally no idea how much do you eat. People can miss-judge their daily intake by 1000 calories easily.
Most people simply confuse calories with being full. It has nothing in common. You can eat 400kcal and be full and 1500 and be hungry.
There’s no other way to gain or loose weight, but to track calories.