Trying to make do with horrible code for long periods of time, suffering through bugs in every change... Until we finally rewrite it. I've never regretted a rewrite, and it has always ended up significantly better than the original (possibly because the horror threshold for rewriting is high, so that's not saying much).
A large, extremely messy code-base can make even trivial changes unsafe, let alone large changes.
The effort required not only to figure out what point B is, but also how to safely reach it from point A is immense. Much much harder than a rewrite.
However, it is a very good idea to meticulously read the bad old code and write down a list of things it handles -- to make sure none of it is missed in the rewrite.