NB: I'm not talking about selling a product for build pipelines, I'm talking about allocating excessive resources into internal tooling pre-traction.
If you are using something like Google app engine or heroku, all that autoscaling and other devops stuff is taken care of for you. There is no reason to go without.
The very first version we launched publically, we checked out, compiled and deployed through SSH directly on the servers. My first monitoring tool was htop in 3 screens plus some CloudWatch dashboards. A lot of credentials and endpoints were hardcoded. Not because we didn't know better, just to save time.
The number one thing pure techies completely get wrong is opportunity cost. Especially in the beginning, almost nothing beats time to market to validate product/market fit.
Doing awful things to rapidly try out an idea and see if it warrants further investment is good, as long as you kill the ugly as soon as you understand that your POC is good and you are going to invest further or you know it's not a wise place to invest further.
But there are definitely places where you know you're going to need sanity no matter what. This is anything you know you're going to do a lot, especially under a time crunch. For SaaS work, deployment is one example. In these cases, it's worth investing in building a low friction solution that meets your needs.
Top-talent engineers are going to be a lot more reticent to join a startup where the systems are constantly on-fire and held together by duct tape. The first few hires set the standards and corporate culture, so this can have a real impact on the company's long-term viability.
Once you, as a founder, have the luxury of hiring (presumably because you've done step one correctly), your time is then a choice between selling and recruiting, and in the early stages of that, recruiting isn't going to be much more than trying to convince good people you already know to join your team. These people aren't going to say no based on your CI system.
Having made the mistake myself (multiple times), OP is 100% correct, and it's one of those messages that can't have any nuance, or people won't do it right. It's a treacherous vortex of failure for programmers-turned-founders: coding is much easier and more familiar than selling, so you find all sorts of reasons to write code when you should be talking to customers.
Ship it
I don't care if you're installing it by retyping the code into your server (hyperbolic example, sure), but ship it.
Can you get payed? Can the customer use the product, even if it's a basic version? Ship it
The rest is fluff. Of course do worry about them, after you start having money go into your bank account.
It's fine to start this way but people have to realise they will need to put some structure in place. At some point you'll struggle to get new features out on time without a ton of bugs.
That said I think that investing in a minimal, quick and dirty CD system can be valuable for very early startups that need to move very fast on the road to product/market fit.
Tooling is tooling, not a religion. But if you've ever looked at the regimen of an elite professional athlete, you'd be hard pressed to discern it from a religion. It is necessary, but insufficient, and I don't think most people get that.