The model used in the post (LeNet) is very small compared to what we typically think of in terms of modern deep neural networks. The LeNet architecture itself was published in the 1990s and was designed for character recognition so this is a natural application of it. kNN would work for many characters in this example, but you would run into a problem with overlapping text. kNN on raw pixel intensities requires a nice segmentation of the ROI. More image processing/segmentation/morphological operations could be applied to help in that case, but given a small network architecture that will naturally learn these filters a tiny CNN works well with little preprocessing.

You might be right. But who cares if using a deep neural network takes 15 minutes? There’s also no need to use a chainsaw to cut down a Christmas tree, but if you’re already holding a chainsaw...

When someone solves a problem with a thing and you're not that impressed, the best response is "cool" or "learning new things is fun" or, like, nothing.

Telling someone they didn't need to do it with that tool is presuming that a) you know why they did it, b) that you are able to judge the suitability for that of that tool for that purpose and under those constraints, and c) that there is no one else who has ever needed to use that exact tool to solve that exact problem under those same constraints.

While KNNs are simpler conceptually, DNNs are fairly easy to put into practice, thanks to Keras and other libraries. A KNN approach would likely involve more code - for things like Data prep

The advantaged of Dnn is that they are easy to understand, very well supported, solve a wide variety of problems, and gpu costs are very rapidly coming down. Plus you can use them to play games!

very likely, something like PCA + KNN does well on mnist. I think what will start to get tricky is rotation invariance.