It's important to have an order of magnitude as a goal. In general the size of a company in dollars correlates with their impact on the world. The best talent is interested in making a big impact on the world.

2x or .5x won't make much of a difference, but if you are thinking 100x or 1,000x smaller then it will have an impact on your business in the early stages. If your goal is to make a $100M company you aren't likely thinking big enough to attract good talent and partners.