See his response on twitter. https://twitter.com/geoffreyhinton/status/165299357072121037...
97 karma · joined January 21, 2015
They also have professional service arms that are similar to the rest of the industry. Handful of senior people and an army of junior engineers that bias towards velocity over quality (i.e. take shortcuts that can lead to data exposure and other issues)
The fact that shell commands were run repeatedly means that the data ends up in the OS buffer cache and basically in memory.
I'm not discounting that CLI is faster than Hadoop by an order of magnitude on small datasets. Nor will I dive into Hadoop vs CLI. The answer to all that IMO is that it depends. And in this case, it's not well warranted.
What I do take exception to is the Fox News style headlines that are disproportional to the truth. EMR != Hadoop.