I think the term "Hadoop" is becoming almost meaningless. It seems to now be more of a pointer referencing a basket of distributed processing technologies that run on YARN/HDFS. Agree completely with having multiple technologies to solve every problem, that's one of the most confusing parts to learn.

My own perspective is that there are lots of businesses that haven't yet needed the capabilities provided by a platform like Hadoop, but they likely will in the future. So the market may be saturated based on current needs but that market will continue to expand. Whether it's Hadoop (YARN/HDFS/etc.) that wins that market share or some other stack like Spark/Mesos remains to be seen.