There's no inherent entitlement or human right to give your children your money, or to not be simply turfed off what ever land you are using when society decided there's a better use for it, for a competitor simply stealing your inventory, expecting protection from thugs taking your business etc., etc.
Because wealth begats wealth, there needs to be certain checks and balances, minimum wages are one of them, inheritance taxes and capital gains taxes are others.
For that, you get the protection of strong laws, an infrastructure you paid almost nothing towards, legal protections for your property, protection from foreign governments, access to skilled trained workers you didn't pay to educate, etc.
Your argument is circular, Fred is wealthy and can afford to speculate, therefore Fred deserves more wealth.
But Fred is only wealthy when everyone else buys into the system, otherwise Fred would soon be Dead Fred.