I would ask you, how many of you have actually tried to teach kids how to code? How many of you have the experience to back up your assertion that it's simply "too hard" and only a select few should have the privilege? I would imagine a handful and I would be surprised to hear anyone who has had that experience tell me it's been a negative one (if it was I would sincerely love to hear it and find out what issues you had).
For the past six months I've been teaching kids, at the high school level, with little to no background in programming and all of whom Engilsh isn't even their first language (it's an internationally diverse school), web development skills (js/html/css). What I've found, which is pretty much what I expected, was a varying degree of proficiency, some kids have gotten it really quickly and some have needed a lot more help. But overall I've been amazed at their progress and am totally convinced that this is something that needs to happen in more schools at a wider scale.
There are real problems in trying to tackle this problem. I've met actual teachers that have been involved in trying to teach CS curriculum and there is definitely a lack of professional training among them. What they could really use is the tech communities help and support, not their derision and dismissal, it's so easy to view this from a cynical perspective instead of trying to actually get something done. If any of you are actually interested in trying to teach and see for yourselves whether this is worthwhile and you live in the NYC area take a look at ScriptEd (http://scripted.org/). Their focus is specifically in promoting CS in socioeconomic under-represented groups. This is where I volunteer but I know there are a few other organizations you can volunteer at as well, their names escape me at the moment but you can attend the CS NYC meetup to learn more (http://www.meetup.com/CSNYC-Education-Meetup/). If you don't live in the the NYC area not sure of what other resources are out there but I'm sure at least SF has similar initiatives, and probably a few other large metropolitan areas.