Lethalman

You mean only one way from N machines to a single machine? Yes. When other machines add/change any file it will be uploaded to that single machine.

Vagrant could run inotify+rsync periodically from both ends. It works for few files. SFS is when doing an rsync of a whole tree is not an option, in case of a big number of files.

That's true. I've tried ceph, swift and riakcs.

Our requirements were two servers, the storage is several terabytes, and losing data because of an unknown bug in the filesystem wasn't an option.

Such filesystems are quorum-based and certainly need more than two servers with terabytes of data. More than what we could afford. Also if you lose quorum, the system hangs to ensure atomicity. But we don't need such properties.

Also after some testing, those filesystem resulted about 5-10% slower than traditional filesystems, with few nodes of course. They are designed to scale with many nodes.

Glusterfs is for synchronous replication. Glusterfs asynchronous replication, at the time of writing SFS, only supported master-slave.

Good, thanks for clarifying.

lsyncd uses inotify. Also, it does rsync (or other backends) of the whole tree instead of single files AFAIK, but that's of course not a stopper as it would need few changes to implement.

In addition, I don't think lsyncd has been designed for an active/active setup, rather for a master-slave setup.

Yes, we've thought a lot about using inotify, our first prototype was also using inotify.

- Our system needs to cope with millions of directories. Millions of directories for inotify mean a lot of structures in the kernel. For large numbers it can also mean gigabytes of ram. Add to that the mapping in userspace of file descriptors for rename operations.

- Using inotify would take a lot of time at startup to recurse into millions of subdirectories.

- inotify will not automatically watch new directories. You have to list all the files right after the creation of a directory and watch the subdirectories, etc. Not a problem at all, but way simpler with FUSE.

- If your system is not able to keep up with inotify events, you miss events because the kernel cannot buffer all the events. For us it's better to slow down the system but never miss an event.

- inotify is attached to the original filesystem. That means it's hard (not impossible) to handle loops in an active/active replication setup. Whereas with SFS, replication is done on straight to the original filesystem.

- If the inotify application crashes, you lose events because software keeps writing to the filesystem. If SFS crashes the mountpoint is unwritable by the application which reports an error and can switch to a different storage.

Some choices as you can read depends on the requirements, our requirements where not met by inotify.

I'm an employee at Immobiliare.it and yesterday we've released for the first time some internal software to the public on github. Just wanting to share our work :)

To my understanding, Ubuntu Core is much similar to the idea of Lennart: http://0pointer.net/blog/revisiting-how-we-put-together-linu...

I don't feel saying them as either bad or good compared to Nix. Until you try a system, there's little to say only from the theory.

We'll see how Ubuntu Core with snappy performs, and this question will be more relevant in a few months.

Nix is not perfect but certainly has a long story maintaining this kind of packaging.

USE flags are just different inputs to the functions that build stuff in nix, which is most probably even more flexible in nix. Nix is mixed binary and source compilation when there's no cache.

For using with nix-env, there's still -I, or -f. E.g. nix-env -f ~/nixpkgs -iA thepackage.

Yes, in general you can't say "what version of X am I using?". You have to ask: "what version of X, the derivation Y is using?". You can see the contents of package X on your system, there's no index provided online for that. Also "contents of package X" is too general for nix, it's "contents of package X built with inputs Y", you may have the same package X built in different ways in your nix store. It's not impossible, there's the need of a service that provides information on what packages provide a certain file.

Hi. I'd like to run NixOS on kimsufi as well. Do the servers work well? Is the cpu what they say on the page? Do they provide kvm?