The problem is you need capital to begin with, and the median American barely has enough on hand to cover an emergency, nevermind build significant investments.

Many of us might be fortunate to have well paid jobs, and the ability and capital to invest, but it's understandable how someone could view the ability of those with capital to generate further capital without any labour to be unfair.