Yes, but right now there is no technique not making use of Deep Learning that is remotely competitive. It really is that clear a distinction.
Right, that's really what I was asking. Thanks.
Lots and lots. NLP covers a very wide range of techniques. Even some 'fancy' techniques such as sentiment analysis can be done relatively simply without neural nets (eg VADER). On the low end tokenizing is a NLP technique eg. r"\w+"
This depends on the task. The non-neural techniques aren't making huge leaps in performance though the way neural methods are. There are tasks where traditional methods still excel or are at parity with neural methods, but the metrics haven't moved much in recent years so you don't see them in the news.