"It should be possible to avoid the userspace to kernelspace transition in the client by using LD_PRELOAD with a shim library as well."
We (I'm a GlusterFS developer) actually did this for a while once, and people have recently started talking about doing it again. With GFAPI it shouldn't even be that complicated, but there are still problems e.g. around fork(2).
"The Ceph distributed filesystem has an in-kernel client which is upstream."
...and it's probably not a coincidence that the file system component is the one piece of Ceph that still hasn't reached production readiness. Development velocity counts. The fact that it uses FUSE has rarely been the cause of performance problems in GlusterFS. More often than not, the real culprit is (relative) lack of caching, which is fixable in user space.
"XtreemFS and MooseFS are designed for a different use-case (WAN filesystem)"
Pretty true for XtreemFS - for which I have the utmost respect and which I promote at every opportunity - but MooseFS targets exactly the same use case as GlusterFS. OK, a subset, because they don't have all the features and integrations we do. ;) I'd also add PVFS/OrangeFS, which is contemporaneous with Lustre. It doesn't use FUSE, but has its own user-space "interceptor" which is equivalent.
"Lustre runs the majority of the world's supercomputers."
It runs a lot of the world's biggest supercomputers, because those people can afford to keep full-time Lustre developers on staff to baby-sit it. Not ops staff - developers to apply the latest patches, add their own, etc. I spent over two years at SiCortex trying to make Lustre usable for our customers. At that time, I believe LLNL had four Lustre developes. ORNL had slightly less. Cray, DDN, etc. each had their own as well. When it works, Lustre can be great. On the other hand, few users can afford to devote that level of staff to running a distributed file system. Those that can't will find themselves in the weeds with MDS meltdowns and "@@@ processing error" messages all the time.
Because of this, I'd say it's Lustre that's not really targeting the same market or use case as GlusterFS. We never encounter them head to head, in either the corporate or community context. The "performance at any cost" market is a hard place to make a living, and it barely overlaps at all with the "performance plus usability" market.
"It's unclear to me whether Red Hat will want to support two distributed filesystems going forward"
Why not? They've supported multiple local file systems for a long time, and there's an even bigger overlap there. When you look at both Ceph and GlusterFS in terms of distributed data systems rather than just file systems specifically, maybe things will look a bit different. Now we're talking about block and object as well as files. Maybe we're talking about integrating distributed storage with distributed computation in ways not covered by any of those interfaces. We're certainly talking about users having their own preferences in each of these areas. If there are enough APIs, and enough users who prefer one over the other for a certain kind of data or vertical market, then it makes quite a bit of sense to continue maintaining two separate systems. On the other hand, of course we want to reduce the number of components we'll have to maintain, and there will be plenty of technology sharing back and forth. Only time will tell which way the balance shifts.