It really depends on what the workload is and if performance or scale are priorities. Our team and the products we build come from the HPC side of the fence and we've been engaged with all sorts of large businesses that know their performance requirements (think 100s of GB/s read bandwidth), and it leads them down the path of either HPC filesystems or newer NVMe-oriented filesystems. Does 'AI' or 'data science' always mean 'high performance'? No, but it definitely can.

Everyone's data volume is different as well...working a proposal for several petabytes of NVMe storage right now. It's not everyone, but use cases are out there for large volumes of high performance storage.

Fair point on the ability to swap the hdfs interface trivially, I hope it's that simple everywhere. Another issue we run into is that these companies have invested heavily in these Hadoop clusters, and would prefer to continue to get some use out of the mountains of HDDs that are captive in these environments. So tiering/HSM functionality is another facet of the issue that these environments will anchor for quite some time.