Totally. It often becomes a trade-off of doing an alright job, e.g. approximately, optimizing the loss you're really interested in vs. doing a great job optimizing a tangentially related but much less appropriate loss.