I don't think the main problem with operator overloading is "abuse"; I think it's that there's absolutely no way to know what a given clause of code might mean, even if the overloading has been defined by excellent coders.
The risk is that programmers might start using symbols in a way that is not obvious, but what is obvious is highly contextual. I'd prefer it not be the language designer who decides which operators are obvious enough to be allowed in my context.