To a large extent that depends on your definition of empire. If an 'empire' is defined as a multi-billion dollar company then I agree with you, if it is anything over 10 million then there must be 10's of thousands of them.
I'm pretty sure my own company does not qualify as an 'empire' by any standard, but it keeps me free, off the streets and pays the bills, I couldn't wish for more. I wouldn't exchange it for a boring job in investment banking for whatever compensation you put on the table, there's more to life than money, as I've found out the hard way.