It's possible the problem isn't speculation, but the rate of return on capital vs the rate of return on labor.
If capital dominates, you're going to run into scenarios where it's fundamentally impossible to ever afford scarce goods that generate returns, because someone who started with more capital will now have an even larger delta with you, and is therefore able to outbid you.