The thing that is unusual about RSA is how many people /kind of/ understand it. Crypto people who dislike RSA say that this leads to a proliferation of terrible RSA implementations, and that it is therefore more dangerous to use than eg ECC. Crypto people who like RSA say that its relative accessibility makes it a more popular target, and that in the absence of a catastrophic break the more-studied cryptosystem should be assumed to be more secure.
Personally I've spent some time recently with badly implemented ECC, and I don't think the mistakes being made there are fundamentally different from or rarer than the mistakes you see in poorly implemented RSA.