close(0);
fd = open(“/foo/bar”, …);
// fd is guaranteed to be 0
If you made close() just dispatch an asynchronous operation and not wait on the result, then the code above would break. Any code that uses dup() likely has code that expects close() to behave that way.The other issue is that close() can return errors. Most applications ignore close errors but to be a robust solution you’d need to ensure the target application ignores those errors as well.
[1]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/V...
On a multi threaded system that isn't guaranteed is it? Meaning, another thread could call open in-between your close & open.
Go's random map iteration does not apply here. Not only is this not an iterable map, the kernel has no problem providing this insertion guarantee so adding additional costly randomization has no benefit and just burns additional cycles.
Go would also be better off without, but they are catering to a different audience and different degree of specification, and apparently need to actively deter developers from ignoring documentation.
In addition to violating POSIX’ lowest hole rule, it would break select(2) (more than it’s already broken).
While you could shard the file descriptor tables for CLONE_FILES processes such as threads, you would likely complicate file descriptor table management and harm the much more important read performance (which is currently just a plain array index and pretty hard to beat).
You could also juts create your processes (or threads) without CLONE_FILES so that they get their own file descriptor table. ------
The fdtable can be seen here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin..., and alloc_fd and __fget can be found here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin....
With a sufficient number of users of an API,
it does not matter what you promise in the contract:
all observable behaviors of your system
will be depended on by somebody.
I guess that we got this "lowest available" rule because that's what the first implementation happened to do (it's the obvious thing to do if you have a single core), then someone 'clever' noticed that they could save 3 cycles by hard coding and reusing the fd in their IO-bound loop, and anyone that tried to implement fd allocation differently was instantly met by "your OS breaks my app", and thus the first implementation was permanently ossified in stone. To be clear I'm not making any historical claims and this is pure speculation."Stupid developers should have rtfm humph" is not a useful position because it ignores this behavior ossification.
The Go map example is actually very relevant, it's an "anti-ossification" feature that makes the behavior match the spec. If the spec says iteration order is not guaranteed, but in practice people can rely on it being the same in some specific situation (say, in a unit test on a particular version of Go) then the spec is ignored and it breaks people's programs when the situation changes (e.g. Go version updates). This actually happened. Instead of giving in and ossifying the first implementation's details into the spec, Go chose the only other approach: Make the behavior match the spec: "iteration order is not guaranteed" == "iteration order is explicitly randomized". (They do it pretty efficiently actually.)
Maybe I'm getting old, or maybe I find the permanent useless change tiring. I'm looking at GNOME, Android, Windows in particular.
Should you want to have per-process file descriptor tables, you can do just that: Just create a process without CLONE_FILES. You can still maintain other thread-like behaviors if you want. I doubt you'll ever sit with a profile that shows fd allocation as main culprit however.
> If the spec says iteration order is not guaranteed, but in practice people can rely on it being the same in some specific situation ... This actually happened.
If Hyrum's law held, the API would already be "ossified" at this point.
Instead, the Go developers decided to make a statement: "The language spec rather than implementation is authoritative". They broke this misuse permanently by making the API actively hostile, not by making it "match the spec" as it already did.
While one could interpret the current implementation as "anti-ossification", I interpret the action as anti-Hyrum's Law by choosing to break existing users in the name of the contract.
Referring to exactly the point you cite, the standard seems to be making no strong statement at all. It says to allocate from the lowest fd but that calls which may return multiple fds do not need to guarantee they are adjacent. I always took this to mean the values should pack downward and should not be e.g. allocated randomly, though it never seemed clear to me why, as the standard seems to be planning for multithreaded code.
So you are interpreting it one way, but the same statement seems to imply that fds are not meant to be introspected and should always be taken at face value from a call that generates a valid fd.
If the program has fd 0-3 and 5 open, socketpair should return 4 and 6, which are not adjacent. If socketpair is called again, while close(N) (N < 7) is being called in a separate thread, you could get {7, 8}, {N, 7}, or {7, N}, depending on kernel and timing details. All of those returns fit the requirement that the fds be allocated lowest first, but may or may not be adjacent or in absolute order.
The reason for this requirement is that early versions of Unix did not have dup2(), only dup(). It has nothing to do with multi threading as this predates pthreads by more than two decades. The shell (sh) makes use of the lowest numbered property to redirect standard in/out/error when setting up pipelines:
int pipes[2];
/* ignore errors */
(void) pipe(&pipes);
if (fork()) {
close(0);
/* guaranteed to return 0 */
(void) dup(pipes[0]);
close(pipes[0]);
close(pipes[1]);
exec_child();
} else {
close(1);
/* guaranteed to return 1, we know 0 is taken */
(void) dup(pipes[1]);
close(pipes[0]);
close(pipes[1]);
exec_parent();
}
Code like this exists in literally every POSIX shell. Anyone saying code like this isn’t common has no idea what they’re talking about.