It all stems from the desire by stakeholders (who are paying the money) wanting to know when something is gonna ship, before committing the money.

I think it should just be accepted that it is not possible to know. Unlike building a building, you cannot know. It is possible to do a feasibility study, or a prototype, and time box it, but that is as good as it gets.

The stakeholders don't want to hear this unfortuantely. They refuse to believe it isn't possible to know ahead of time how much work is needed. In order to appease, the engineers and managers have developed a set of "practises" such as agile, but in the end, it is a lie.