That's a great point.
One thing I've always done is if I slip up and eat a lot more than I wanted to during the course of a day (e.g., big lunch, big dinner) I'll go ahead and keep eating a whole lot the rest of the day.
I figure my body can only store so much fat in one day so any additional calories won't hurt. I've never seen this idea proven or disproven anywhere but logically it seems to make sense to me.