Even in the collocation facilities google was developing this skill for example by installing their infamous corkboard servers with the drives attached by Velcro.
A lot do have a delusional expectation that building on AWS to handle that expected 10x overnight growth that they dream of justifies it, but that "just" justifies good caching and the ability to spin up frontends on AWS or similar if neeeded. Which ironically makes you likely to spend less on your dedicated hosting, as if your setup is ready to spin up cloud servers when the load goes too high, you can afford to get much closer to the wire before you add more dedicated hardware for your base load.
For most people, renting a dedicated server (there's a huge number of providers that charge month to month, with no commitment) will come out far cheaper for anything they use more than ~8 hours a day on average.
To me, if a startup puts everything on AWS, it's a sign they have poor cost control.
I noticed at silcon mikabout that prity much all the start ups where using aws - I suspect that 90% of them would struggle if they had to set up their own colo.