I don't agree with the assumption that we need to ditch X11 entirely. Implementations are different from the spec, and a common complaint I have is that certain implementations are gathering code smells like nobody's business.
You can fix both an implementation and a spec by continued review and revision, while making choices to either remain backwards compatible or breaking that compatibility with newer revisions. I don't think Wayland is a good solution to this kind of problem, despite it being touted as one in many circles I've witnessed.
The competition is nice, though! I have yet to give Wayland a try, but it's on my to-do list some time on a fresh machine.