Look at log4j for a recent example on what unwanted complexity does. For something matching the subject matter at hand here, look at the various image parsing attacks that have gained large notoriety. Now force multiply by having PostScript, an actual programming language, in the path.
Over time, the old format may potentially be removed if it falls in disfavor, or at least only be accepted in more and more restricted contexts, but this works better in closed systems of course. I just finished excising a complex format in favor of a much simpler one, and since it was internal to the organization cutting out the old one did not cause anyone problems. For browsers, I don't know. Deprecating Java Applets and Flash seems to have worked, but those are maybe "heavier" examples than SVG.
As for comparison to existing vector formats, I don't have the necessary domain knowledge there.