2x or .5x won't make much of a difference, but if you are thinking 100x or 1,000x smaller then it will have an impact on your business in the early stages. If your goal is to make a $100M company you aren't likely thinking big enough to attract good talent and partners.
There are a lot of talented engineers that prefer working for startups than large corporations. There are not enough talented engineers to staff all of the start ups. It's a competition for companies to attract the best talent.
Microsoft's old mission statement: (Paraphrase) To put a computer on every desk and in every home.
Amazon's mission statement: "...build a place where people can come to find and discover anything they might want to buy online"
These mission statements don't specifically state they want to build a $100B company, but it seems pretty obvious that a company that does the things in their mission statement would be worth $100B.