I found that the most time consuming and expensive was turnaround time. If you have a bug in your software you fix it, compile, and go. 10 minutes of work. In hardware your turnaround time can be a week or more if you have to solder around on a breadboard to check changes or bugs.
For the record we managed to build a display device using RGB LED's, a FPGA to control them, a microcontroller to talk to a mobile phone unit that would be able to send and receive updates from a central server, and of course the server that controlled the displays that were scattered around the country in around 1½ years using roughly $150.000. It was an advertising network - take the display, plug it in to a power outlet, and it would wake up and receive news and ads from the central server over the mobile phone network.
also, you can buy fpga h/w approx 100k gates etc. for < 1000$ (cheaper if you are student, and can get it via some univ. program). design, and simulation tools are also available for free (with periodic registrations etc.) the only (slight) pain point being that the s/w is generally windows only.
Unit costs matter in hardware though, so first thoughts should tend more toward markets and demographics -- "what problem should I solve... for how much?"
A new Ferrari (or a new Prius) would solve my getting-places problem better than the POS I drive now. I could buy a new Tesla Roadster, which would be better than either one, but that's even less affordable.
Of course, since a hardware startup requires some hefty NRI that you'll probably have to borrow from someone else, you might want to start by polishing up your resume.