Yeah I would have loved to see HDFS get really scalable metadata management. I remember hearing about LinkedIn's intentions to really do some significant work there are the last community event I attended, but from their blog post this week it doesn't sound like that's happened since the read-from-standby work [1].

Kerberos (quite popular on big enterprise clusters) is really what makes it hard to get data in / out IMO. I see generic Hadoop connectors in A LOT of third party tools.

[1] https://engineering.linkedin.com/blog/2021/the-exabyte-club-...