What's the intended function of the previous functionality? Didn't double-tapping zoom in and out to a specific section? What problem does the delay solve that isn't present on unscalable viewports?
Yes, the whole architecture would need to good undo-support.
In any case, once you start delaying things you ruin the original point of getting rid of the click delay - to make things faster!