I've been keeping millions of files in single directories on XFS without problems. The big upside is that you won't need to learn any funky/new management tools.