Too often real work begins to get something concrete for stakeholders to react to (the general point of Scrum & Agile) while losing sight of the ultimate goal because it was never really defined very well to begin with. That just means more rework. Some rework is unavoidable, but so much of it is.
You can control which mountain you want to climb and you can and should select which face of the mountain makes most sense to use as your path up it, but have a way to move laterally to a safer path up if the weather goes south.