Ceph doesn’t really nail the “I want to boot off this thing” use case. It would be interesting to try, though.
I mean, sounds like a house of cards but, should be possible?
You'll still want a fast network (I'd recommend 10gbe on the server at least, 2.5gbe on clients though 1gbe will work you will notice it bottleneck in bursts) but that won't be any different than any other network booting/rooting process
Or the S3-fuse route if you just want to geek flex.
I’m sure it’s possible. I don’t think this is quite what Ceph is intended for :)