Social security is more of a forced 401k. You will get back the money (in theory), but yes, it's not progressive and impact the poor more than the rich. It's a bit of an iffy one. If the social security system rolled up into federal income tax, it would be much more progressive, but it would likely receive significant pushback in the political space (not knowing its history, I assume it was structured as a separate thing exactly for that reason. I should read up).
Still, the point still stand: federal income taxes are quite progressive, and would surprise most people who wave their angry fist asking for the rich to be taxed more. The problem is mostly at the state and local level.
The top 50% pays 97% of federal income tax.
If you include most types of taxes, including social security, you end up with something a little less polarized: the top 1% have an effective tax rate of a little under 34%, while the poors are around 20%. One could easily argue it's not progressive enough, but it's still not the usual narrative of "I pay more taxes than millionaires".
A a handful of ultra rich abuse loopholes to death to pay very little, and these people are averaged in the statistics (so the average rich person actually pays more than the stats show, if only a little). It's a minority though.