Isn't it self-evident? It harms their standing and reputation, in being cast as in possession of wealth they are not morally entitled to. It's defamation.
Privileged =/= undeserving
Can you show an example of damage done by this sort of "defamation"?