Of course it's not that simple, no doubt someone will be along to mention Goodhart's Law or the inherent problem with essentially quantifying the meaning of life. Still, I don't think people applying logic to their daily lives can be a problem on its own.
In all our mathematical experience, the answer turns out to be that we can only ever define an approximate objective function, and we almost always need some kind of “regularization” to constrain our optimization strategy. And in many cases, an effective regularization practically (which results in better “generalization” technically) might be “early stopping” I.e. not optimizing as hard!
I agree, any sufficiently complex problem is beyond what we can define as a concrete objective function and any of the solutions we have. So instead the best we can hope for is that today is closer to being optimal than yesterday, and tomorrow will be better still.
You say that it gets a "better" solution, so the against some indescribable original objective there's a difference.