I am not sure why you're saying they cheated. I am a layman, but my understanding of that quote is that he refers to a rather large deep neural network, which was trained unsupervised to encode general images (not just cats), had a neuron representing the concept of "cat". You can run the network in generative mode to see how the general concept of cat would look like.
So I am puzzled by your comment - you seem to be talking about supervised learning, but I think the network was trained unsupervised.