At Twitter we don't program using Hadoop directly; we mostly use either Scalding or Pig, languages that compile down to Hadoop code. https://dev.twitter.com/blog/scalding http://www.slideshare.net/kevinweil/hadoop-pig-and-twitter-n...
I believe this is how many other companies use Hadoop as well.
The benefit here is that it's possible to write new backends for Pig and Scalding that compile down to Spark or anything else. And then you have backwards compatibility with all your old big-data code.