> That's what I've done. Hardware RAID + XFS + NFS + backups, done.

Even if this was HA, it doesn't scale to HPC type needs. You end up with a single node bottleneck for all data you need. You also would have some sort of direct attached storage all connected to a single node.

Lustre/Gluster/Ceph/GFS and maybe HDFS (ideally with commercial MapR type NFS access) are probably the only viable options in the HPC use case.

If you have a lot of money to burn Isilon/NFS is an option as well.

I can't emphasize this more. Being able to comprehend the system is my first design choice, no matter how the alternatives are great/stable/fast, ... You are going to pay the price some day as long as you don't know how stuff works or interacts with each other.

> That's what I've done. Hardware RAID + XFS + NFS + backups, done. It wasn't HA, but at least I understood how to get it back up.

Looks like in this case you never actually needed what Ceph brings to the table - for a tremendous cost in complexity (and fragility, from personal anecdotal evidence) you can allow your dataset and -load to grow beyond the physical capabilities of a single node.

The HA argument in less demanding settings ends up being a cargo cult mostly. Your setup will probably produce higher reliability figures than running Ceph, assuming the lack of some serious engineering capabilities.