Yes, I think this is exactly right. The pattern you'd expect to see wouldn't be a small business spawning a large one; it would be one of the principals of the small business cashing out and leaving to start something that eventually got larger.
Ah, but this reminds me. I know three people who have made $500M or more on their earlier businesss who have cashed out and gone on to found new startups. Only one of the three funded the startup entirely out of his own holdings; the other two have other investors involved. I imagine the latter two think that they get more value (advice, connections, etc.) from these other investors (who I think include VC firms) than they give up in equity.
FWIW, the business funded entirely by the one founder eventually failed (I worked there for several years). Arguably, this was a failure of marketing, as we had a small number of wildly enthusiastic customers that never became a large number. Also arguably, had experienced VC firms been in on the deal, they would have made sure that the marketing was done competently.
So I'm suggesting that if you want to start something that you hope will get large, you may be well advised to involve professional VC even if you don't need the money.