Meanwhile, I know what the word desert means, even if I have never been in a country that has one. I know what a cyclone is, even if I've never seen one. We're affected by, and have knowledge of more than our local environment. We read books. We watch TV and movies. Heck, we're having this discussion in a language that that isn't spoken in the country I'm living in. Most people do, in fact, interact in some way with people in other time zones right now. Perhaps not on a daily basis, but at least a bit.
It just occurred to me; if we placed the meridian in the Pacific, very few people would have the date change during their sleep cycle. Sucks to be Niue, I suppose ;)
By the way - I'm not even for abolishing time zones. I'm just spectacularly tired of the straw man arguments against it. It's like saying "we couldn't build a city that's on both sides of the river, because people would drown when they go to work", as if building a bridge would be an impossibility.
There is a perfectly good argument against it btw, which is the one you propose at the end. The change itself wouldn't really be worth it. But that's simply because people are confused by changes. Not because the end result would be worse.