The company I founded has custom hardware and a custom OS (not, e.g. Linux or Windows -- closer to an embedded RT OS), and our "for sale" product is SaaS.
We actually have four different hardware systems, ranging in size from a 1U box that goes on site, to a full rack that's basically a turn-key cluster with no external configuration (just plug in ethernet and power, and assign static IP addresses).
All of our hardware is based on Intel motherboards and chips, especially the E3/E5 series. This eliminates the long prototype cycles, since we use off-the-shelf hardware. The only real prototype cycle for us is choosing components and the case (!), which we did with 3D CAD software and a machine-shop-as-a-service we found on the Internet.
We went this route for reliability and performance (especially, latency), and then getting those two at a reasonable cost. By only paying for the hardware we need, and configuring it to exactly what our product/service needs, we're able to meet our product and business needs while still maintaining healthy margins at a cost our customers can afford. I wouldn't change a thing.
We also face all of the problems you mentioned (how to deal with inventory -- our hardware has a comprehensive 3 week burn in time), how to deal with repairs/replacements in the field, etc, and how to get the capital needed to sustain startup-scale growth. They're real problems.
Like most hardware-based companies, we're able to do this only with outside funding, but at least in our case (the enterprise space), we don't need any up front funding to stay cash-flow positive with slow growth, and we developed a rather innovative way to get the funding that we do need to fund fast frowth (hint: not through VCs). This took a lot of effort at the business-design level, something I suspect most tech startups don't take the time to do. It's pretty much mandatory if you're going the custom hardware route and want to grow fast.
One major positive of going the custom hardware route is the elimination of competitors. No YC-backed company going the standard AWS route can possibly compete with us -- not on cost, not on performance, not on latency, and not on reliability -- not even close. Controlling the whole stack, from hardware to OS to software is why we can do what we do at the price we do it, and grow at the rate we expect to grow.
I'm not sure we could make our approach work in the consumer hardware space today, but in the enterprise market, it's ideal for us.