I'm raising my family in NYC, I disagree that it's one of the worst imaginable places to try and raise a family. There are a lot of activities for children, from the parks, to indoor play areas, to the libraries, etc. You can just about anything you need by walking. The public schools have been improving - you can try for one of the elite gifted and talented schools but even if that doesn't work out, there are many more good school options than there have been in the past. It's certainly not a cheap place to live, but I actually can't imagine a better place to raise a family.
I don't know if there's any more pollution than any other major city. There's no major manufacturing that happens in NYC, so the pollution would mostly be from cars. In general, the city is becoming less and less car friendly - they are seriously considering a congestion pricing program - so I would expect whatever pollution there may be to actually improve. Nevertheless, pollution has never been one of the major complaints of most NYC folk.
Yes, taxes can run high. You can always live outside the city if this is an issue but despite this, people prefer to live in the city as long as they can.
High stress is really more a virtue of your job, not the city.
There are many people who live here and don't want to live anywhere else. As they say, your bedroom is where you sleep - your living room is the entire city.