I think I haven't explained myself properly. It's not that I think the bad refereeing makes the sport better. I don't, and I wish it were fixed.
What I do think is that, with something so obviously successful, it's pretty hard to know what kind of changes might break it.
To use a simple analogy, assume you're a new programmer going over some very old financial software at a bank. The thing is written in 30-year-old COBOL code (and written badly at that). I'm guessing most programmers would cringe at such a system, not understanding why it isn't being "improved" by rewriting it in a better language (and there are good reasons too, like being able to easily add new features, etc.).
But as you and I both know, it's probably not a good idea to change it. Not because COBOL doesn't suck, not because there are good reasons to rewrite (there are some good reasons), but because it already works, and no one wants to mess with that.
P.S. Having said that, I personally do wish they'll change something to make the refereeing more solid. I don't think it will make Soccer any worse, even though I can't be sure. More importantly, I'm pretty sure that Fifa is going to cave on this issue withing 10-20 years, because of public pressure, and allow some form of instant replay.