thatcks

The various Illumos distributions routinely upstream their changes into the master Illumos repository ( https://github.com/illumos/illumos-gate). I know that ZFS on Linux (https://github.com/zfsonlinux/zfs) routinely integrates those changes, as well as making some of its own and I believe pushing them upstream to Illumos. When I've looked at FreeBSD commit logs, they too are pulling in ZFS changes from Illumos on a frequent basis (but I know less about any independent development and upstreaming).

My strong impression is that the ZFS code base is a sufficiently big and tangled thing that no one wants it to fragment. With bug-fixes and improvements happening in Illumos, ZoL and FreeBSD both want to be able to incorporate them on a regular basis and to push their own changes upstream to reduce the maintenance burden of carrying those changes.

Rules of thumb involving more swap than RAM probably date from decades ago, when Unix virtual memory systems were sufficiently primitive that the total amount of virtual memory you could use was just your swap space, not swap space plus (most of) RAM.

(The limitation came about because the simple way to handle swapping is to assign every potentially swappable page of virtual memory a swap address when you allocate it in the kernel. Then the kernel always knows that there's space for the page if it ever needs to swap it out and you're never faced with a situation where you need to swap out a page but there's no swap space left.)

Swap space is only partially related to virtual memory overcommit, and virtual memory overcommit is extremely common and almost unavoidable on most Unix machines. Part of this is a product of a deliberate trade-off in libraries between virtual address space and speed (for example, internally rounding up memory allocation sizes to powers of two), and part of this is due to Unix features that mean a process's theoretical peak RAM usage is often much higher than it will ever be in reality.

(For example, if a process forks, a great deal of memory is shared between the parent and child. In theory one process could dirty all of their writeable pages, forcing the kernel to allocate a second copy of each page. In practice, almost no process that forks will do that and reserving RAM (or swap) for that eventuality would require you to run significantly oversized systems.)

What is wrong with the standard procedure is that once upon a time the standard procedure that everyone knew and coded to was steps 1 through 4, without step 5. Then Linux ext4 people decided that step 5 was necessary, or rather that they could optimize ext4 to delay and lose some writes if you left out step 5. Which lots of people did, and so some people unexpectedly lost data and were unhappy.

If we have only a standard procedure and not an actual agreed on standard (whether or not it's a formal one), we are forever vulnerable to the next clever filesystem developer who decides that obviously they can get away with adding a new step 6, or 7, or 8 in some situations, under some interpretation of the POSIX or Single Unix Specification wording, or the Linux manpages, or whatever.

(I'm the author of the linked-to blog entry.)

It's how the SSH protocol itself works. When the client connects to a server, it sends the (remote) username and then a series of public keys. If the server will accept the current public key, it asks the client to authenticate with that key to show that you hold the private key; if it doesn't, it says 'try again'.

This means that a malicious client with just the public key can probe to see if a server will accept a given username/public key combo. If the server does, it will challenge you to authenticate that you hold the private key (which you'll have to fail, since you don't have it).

Exposing shared public keys creates an information leak; it allows attackers to probe for valid and usable username/key combinations on your servers (and thereby discover valid usernames). This may be a trivial information leak or it may be one that you consider important; it depends on the situation.

Not sharing public keys between different services and contexts (eg not using your Github keys for anything else) mitigates this risk significantly.

A note: SIGHUP is far from obsolete in its original purpose. We may have stopped using modems but people still log in to Unix machines in ways that can get disconnected. If you SSH in to a machine and your SSH session is cut off by a network issue, your shell (and any command you have running) will get a SIGHUP.

Our experience is that NFS works excellently in the environment it was designed for (a LAN environment of Unix machines administered by the same people). I still haven't seen a better Unix remote filesystem (and most of the modern distributed filesystem efforts are solving different problems, as seen by the fact that they don't support full core Unix semantics).

With that said, the environment NFS was designed for is increasingly uncommon. NFS is not necessarily a good fit for other environments, even if they look superficially similar.

In our case we run servers and then have thin desktops (using LTSP) that actually log in and run their sessions on the servers. The servers carry our entire password and group file because everyone can log in to them.

In general, in any NFS based environment all machines that NFS-mount common filesystems need to carry a full collection of login and group information so that things like 'ls -l' work on the NFS filesystems. If you want everyone to be able to log into every machine, they also need some sort of password information as well (either by distributing shadow files or by various other means).

The problem (for large sites) is the CPU usage required to repeatedly re-parse and re-process the group file; this is apparently not insignificant for large group and password files. IO isn't an issue because as you note, it'll be cached.

I think there's been a misunderstanding. We (the people who ran into the accountsservice problem) don't have a Canonical support contract. hp50g, who commented upthread, is the one who had the Canonical support contract.

We did 'fix' these issues as soon as we managed to track down why the daemon was using up so much CPU time on our machines. I wrote my entry partly because I was annoyed at how terribly badly thought out the code was and partly to warn other people.

(We have never tried to feed the 'fix' upstream because our fix simply hardcodes some answers. I don't think it's even possible to fix the code to do what Ubuntu apparently wants it to do, although it could be less catastrophically inefficient.)