> If you've fixed the design to 320px that's about half the width of my phone screen and a quarter of a portrait tablet screen.
Take a look at Twitter's Bootstrap. It has a variety of different responsive design levels, not just one at 320px. Great example of how a design can fit many different devices.
> Mobile Safari doesn't automatically zoom because Apple were to rushed to take that feature out, it auto zooms because Apple went out of their way to add it.
Yes, and the reasons they did are good ones - many sites don't have a responsive design.
You know what else Apple went out of their way to add? The ability for a web developer to disable the zooming feature!
> ... maybe I'm in a moving vehicle or using it at night in low brightness and can't see well, or without glasses or when tired or while holding something else and viewing at a distance or anything ...
A trade-off I'm willing to deal with. There are accessibility features in iOS and Android for most of those situations, so I'm fairly comfortable making the site work best for most people instead of the small number edge cases.