The need to scale is inevitable if you consider 2 trends: startups have become mainstream and the internet is orders of magnitude bigger. When YC applications consisted mostly of PG essay readers the chance of success per company was higher. Also some niche ideas can now work because of a more developed communications infrastructure, easy payments, better targeting, etc.

And knowing which companies are high quality is almost impossible at the early stages.