Hacker News Reader

Comment by the-rc | Hacker News Reader

That was something else. Most command line flags were untouched from the defaults. What was a problem was GFE's command line. For a number of reasons, it had to list every public service and its config. There were many hundreds of those, probably thousands by now. And there was a kernel limit of 128KB, which was kept at bay for a while with tricks (hacks?), until eventually a proper solution was implemented.

View on HN

Hacks such as: the ability to unpack the command line from a single compressed and base64-encoded arg.

Fun fact about Linux's now-unlimited, demand-paged argv: the JVM was a bit behind the ball on that one. As soon as Linux systems with enormous args limits appeared, `xargs` happily adapted. But the JVM at that time had quadratic command line parsing. So anyone using `xargs java` suddenly had cron jobs (or whatever) that never finished.

What prevents a more general solution like using @flagfiles instead of literally specifying them as arguments?

I don't know. Judging from source blame, flagfile has been in gflags since 2007 at the latest, and 2007 is also when the command line length limit was lifted: https://github.com/torvalds/linux/commit/b6a2fea39318e43fee8...

I remember 128KB being an issue past mid 2007, which is when that patch was merged. I think that the compression trick actually happened a bit after that. Looking at the patch, it appears that yes, it's one of the temporary hacks I was referring to. It did add some breathing room, because it no longer constrains the aggregate of argv plus the environment, but it keeps 128KB as the max size for each flag, which was still problematic for good old GFE and its service list. I can't remember if they managed to implement external flag files first or if they had to resort to --flag, --flag2, as web folks do nowadays to work around e.g. cookie limits (I'm looking at you, oauth2-proxy, embedding massive claims).

My only guess is that it plays badly with borg run somehow but having the borg cli autogenerate a flagfile seems uncomplicated.

Edit: Although there's arguably security implications to letting me push a file from my workstation, but that wouldn't have been an issue in 2006.

As far as I recall, until the plumbing was heavily redone a decade later or so, you could only push (propagate) an updated command line to replicas, not a separate file as well. On top of that, I vaguely remember something about bootstrapping and ensuring that the remaining flags and the flag file always be updated atomically, in lockstep.

For regular Unix folks, imagine issuing a remote ssh command. You can't easily do that AND bundle an auxiliary data file at the same time.

At that point, why not just use them as a library and call them as an API from bespoke command line tools?

Reply on news.ycombinator.com