Well it literally is. If they are talking about Wall St. bankers they have no way to replicate the results of those people. Wall Street can make poorly thought out multi-trillion dollar bets on housing bonds and when they go bust they get bailed out and still get their Christmas Bonuses. There is no similar mechanism to protect a non-institutional investor from similar levels of risk.
If you want to sit on the sidelines because it's too risky for you, that's fine. But don't then complain that others make money on stocks.
You, and anyone else, can buy FAANG stocks, index funds, or whatever.
Many of us might be fortunate to have well paid jobs, and the ability and capital to invest, but it's understandable how someone could view the ability of those with capital to generate further capital without any labour to be unfair.
I've known many people who didn't have any cash on hand, yet lived high on the hog with expensive clothes, new cars, and nice houses. They didn't have any cash on hand because they spent it as fast as possible.