Unlike other businesses, tech businesses are rarely tied to a particular place. They have no plant or buildings, except usually for servers, and their customers usually come from all over the world. This means they're very easy to move.
So start where you are. And if you need to move later on, it's no great hassle.
Depends on personal style also. Do you find it useful/necessary to get inspiration and support from regular meatspace meetups with people with similar interests, e.g. hanging out at a place like the Hacker Dojo? Or are you more the sort of person who gets that sort of interaction from irc, mailing lists, IM, forums, and so on, and doesn't much care about irl meetings?
1) Start with genius, this is a phrase used by Meredith Belbin in his advise about selecting a team. So, I am basically saying go where you will find the talent necessary to implement your start up.
2) Start in a place with lots of early adopters. That is, start where the people are known to try new stuff and be seen as cool if they are at the cutting edge with technology.
Just a few places: Boston, Silicon Valley, London... There will be other places around the world with these characteristics.
- proximity to universities - increases the probability to find talents and people willing to spend money in IT services
- proximity with airport and train hubs - increases the probability of having customers (+ transport => + businesses)
- high bandwidth Internet availability
I don't know what you are referring to regarding health care, but basically if taxes and insurances spending break your profitability, you didn't have a real business to start with.
Your number one spending will be salaries by a huge factor.