Sure it can be difficult, especially if you compare day to day, but if you compare or weekly bi-weekly averages ( both weight and calories ) it becomes way more doable. Are you just supposed not to try since you can't exactly find the numbers each day?

With the approach above you don't really care about acute changes in intake or activity and look at the bigger picture and change the things that are constant and under your control.

It's not simple but it also doesn't mean you should just give up.