- Ceph: Very flexible. Supports many different kinds of replication. Has high overhead compared to local disk (on the order of ~50%) and was (for me) prone to hard to diagnose issues. Can be annoying to setup if you're not doing it on a supported Linux distro with ceph-deploy. It looks like Bluestore (a new on-disk format for data) will significantly improve performance but Bluestore is extremely RAM hungry.
- GlusterFS: Much faster than Ceph but less flexible. Has odd requirements about "bricks" being the same size. Much less RAM hungry than Ceph.
- A bunch of smaller ones I can't recall. Mostly discarded because they performed badly or lacked replication options (I really wanted erasure coding).
In the end I'm simply sharding my data manually. It's not as scalable but it's much more performant.