Hacker News Reader

Comment by jeffbee | Hacker News Reader

Hacks such as: the ability to unpack the command line from a single compressed and base64-encoded arg.

Fun fact about Linux's now-unlimited, demand-paged argv: the JVM was a bit behind the ball on that one. As soon as Linux systems with enormous args limits appeared, `xargs` happily adapted. But the JVM at that time had quadratic command line parsing. So anyone using `xargs java` suddenly had cron jobs (or whatever) that never finished.

View on HN

What prevents a more general solution like using @flagfiles instead of literally specifying them as arguments?

As far as I recall, until the plumbing was heavily redone a decade later or so, you could only push (propagate) an updated command line to replicas, not a separate file as well. On top of that, I vaguely remember something about bootstrapping and ensuring that the remaining flags and the flag file always be updated atomically, in lockstep.

For regular Unix folks, imagine issuing a remote ssh command. You can't easily do that AND bundle an auxiliary data file at the same time.

I don't know. Judging from source blame, flagfile has been in gflags since 2007 at the latest, and 2007 is also when the command line length limit was lifted: https://github.com/torvalds/linux/commit/b6a2fea39318e43fee8...

I remember 128KB being an issue past mid 2007, which is when that patch was merged. I think that the compression trick actually happened a bit after that. Looking at the patch, it appears that yes, it's one of the temporary hacks I was referring to. It did add some breathing room, because it no longer constrains the aggregate of argv plus the environment, but it keeps 128KB as the max size for each flag, which was still problematic for good old GFE and its service list. I can't remember if they managed to implement external flag files first or if they had to resort to --flag, --flag2, as web folks do nowadays to work around e.g. cookie limits (I'm looking at you, oauth2-proxy, embedding massive claims).

My only guess is that it plays badly with borg run somehow but having the borg cli autogenerate a flagfile seems uncomplicated.

Edit: Although there's arguably security implications to letting me push a file from my workstation, but that wouldn't have been an issue in 2006.

At that point, why not just use them as a library and call them as an API from bespoke command line tools?

Reply on news.ycombinator.com