Stop supporting old releases
medium.com
medium.com
If you are doing that, shame on you, that's not how stable and long-term-support works at all. If you backport, you backport on a best-effort basis, not on a must-have basis.
There is 0 sense in supporting old versions when you just backport all the new stuff into them anyway.
>You, in your GitHub project, probably don’t have the time or resources to manage more than one release line of development.
That is certainly true. Over the whole essay I don't find much reason not to have a LTS schedule.
I don't have to backport all the stuff to older releases. My own policy is that any previous release will receive security patches as long as I can be made to care (usually about 1 year and in one case I support a 3 year old release of a project with security patches)
It is not that much effort to patch security stuff. It doesn't take you away from your project for days, it costs you a few hours and some users of your project will be happier for it. That's all I want.
I hit this a lot of with the Rails ecosystem. In many cases, upgrading to the latest version of some gem was more disruptive or riskier than keeping the current version and patching myself. But some devs will also yank old versions making that tricky as well. Rails itself, however, does a remarkable job of providing both patches and backports for security issues.
I'd like to extend a big thanks to anyone that does expend extra effort to provide a solution to existing users. When having to deal with a security issue, nothing's more frustrating than not being able to upgrade immediately because doing so entails API changes or incompatible dependency graph changes.
Besides it being easier, isn't not porting feature patches the entire point of Long Term Support releases? People use LTS because they don't want any unexpected behaviour changes, only security updates.
LTS destroys your codebase. LTS allows mistakes like pushing state into your server that you didn't know was there. LTS makes it much harder to upgrade because by the time you're moving off of, say, Ubuntu 12.04 the whole world has changed. LTS multiplies the number of things that can go wrong. LTS makes people think that the OS is going to be around forever so they don't architect their devices to be OS-updateable. IoT is bad now, but it's slowly getting worse too because half these devices are stuck on some out of date version of linux running some out of date version of Python.
All my servers run Ubuntu 17.10. Rails is on the latest version. Ruby is on the latest version. I don't pin any of my gems and every time I upgrade Ruby I upgrade the latest version of all the gems too.
This is only possible with extensive testing, and mistakes do get made here and there, but overall it's much, much easier handling these issues one at a time. The only thing I don't do is keep current on packages that I install via `apt` because I don't want to risk building from source.
Why the hell would a high level language like Ruby have such deep OS dependencies? Use the goddamn POSIX standard and you'll be compatible to every single important OS that's come out in the last decades, except for Windows which can be made POSIX compatible with a few layers of abstraction.
Sure, this is bad - but if they are to be OS updateable then the OS has to guarantee it won't drop support for things, or increase significantly in size because then it won't for in the flash.
Enterprise customers really like LTS as a concept, but they should be made to pay for it like the article says.
1. Internet connected. 2. Code and forget.
Choose one or none, but not both.
Selling medical equipment that gets hacked by wannacry is not acceptable. It isn't even theoretically possible to promise that the OS won't drop support for things since you may rely on something that is later exploitable. Realistically speaking, unless you're writing code for something crazy like VR, there is a great chance that with tests you can update the code to match the new OS in a week or two.
Seeing fundamental things like glibc or gcc breaking almost every release makes you start to value LTS releases a bit more.
Disclaimer: Im maintainer myself and i feel your opinion is disrespectful towards the people who fix regressions for you (for free, even!).
Also, as I said in the original comment, I only lean on apt for a minority of things. Most stuff gets upgraded right away and you know what? I find and fix or communicate bugs because I'm on the bleeding edge. And I frankly do not see how my using apt invalidates my views on LTS. In the ruby world upgrading is almost never a pain because most of us keep current and test our codebases. I shouldn't be maintaining packages myself, and I do think that if the people behind Debian / Ubuntu / rando packages available took my worldview it would be easier to always be on the latest version and upgrading shit wouldn't be this giant headache all the time.
Common, glibc should break or break something every release. This is insanity.
OS package management doesn't work this way, because, for the most part, for almost the entire history of OSes, developers have had no control over the environment their application is running under. In a lot of cases (enterprise IT; academic/HPC mainframes; PaaSes) someone else has an upgrade schedule for the OS packages, and you don't get any say on when it happens. The OS updates, your package breaks, your problem. You can maybe ask the ops department to hold off on updating, or have your PaaS "pin" your stack (until it becomes deprecated.) Either way, you're now vulnerable to security vulnerabilities while (hurriedly) working to push out the new release so that ops can get back to doing its job as normal.
LTS is a guarantee by the OS distributor that your ops staff can update any package at any time (from the LTS repos, at least), and such updates won't break your deployed custom software. So LTS solves the cases above: enterprise/academic/HPC/PaaS ops staff can just upgrade as they will, without worry that anyone will tell them to hold off.
But the value of LTS goes even further: if updates are guaranteed by the distributor to not break your runtime environment, then you can actually allow OS package upgrades to happen completely automatically and asynchronously to your development release schedule. The OS can just update itself. The OS distributor is now, themselves, part of your ops staff, ensuring that these updates aren't bothering anyone downstream of them. Your actual company doesn't have to care about the OS. It's always going to be secure, because it's going to update as soon as vulnerabilities are discovered+patched, and you can just deploy your code at your own pace without worry.
Now, mind you, this all might sound quaint to people who bake containers for everything and treat said containers' host OS as immutable infrastructure. For these people, OS package security vulnerabilities aren't really something you can let "fix itself"; they're something that requires baking a new version of your container-image, just as app library-dependency security vulnerabilities would be. Relying on OS services rather than bundling a copy into your container-image is a trade-off in this case: you don't get to pin the version of a dependency to ensure its exact behaviour, but in exchange, the distro will attempt to ensure that it will be free of security vulnerabilities while being pretty much the same version you started with.
(Oh, and as a side-benefit, usually LTS package updates don't force system reboots. Recently even kernel updates don't, with fancy ksplice-based patch distribution channels. So LTS releases also mean long individual-host uptimes, which is handy if you're running a service on the cheap and can't afford much horizontal scaling to manage load-balancing under rolling updates, but want five or more 9s of availability anyway. Try getting that using immutable-infrastructure reboot-to-update VM images.)
How do you deal with this in production? Gems get abandoned. Some gems get upgraded faster than others. Some before you're ready.
You get into a race where you either need to fix your code, or fix a gem, or not upgrade. What do you do then? What if you don't have time to fix code to deal with a gem upgrade that ca happen at any time?
When things somehow go sideways I do the same thing: Triage the situation, including taking things offline if needed, make choices that solve the short term situation, run the tests, everything ok push live. Otherwise mitigate the issue by partially removing services (no image upload right now) and getting the rest of them live.
On the server itself I have a second copy of the repo. I install the new version of ruby in that repo and install the gems in that repo. I run the tests there and I check to make sure that Gemfile.lock hasn't changed. All good? Time to go live. Otherwise? Fix things locally first and re-push.
Rich Hickey has a great talk here https://www.youtube.com/watch?v=oyLBGkS5ICk
That said, doesn't work at all levels, but in general it's not nearly common enough in the ecosystem to keep compat consistent. When I think of libraries I love to use, I think mostly of libraries where compat is very well maintained.
Package management systems tend to be working against you here. Bumping major versions for "good" reasons makes it so that your biggest library users just get stuck, because they have so much code in the old paradigm. Real compat changes deserve new names/namespaces, so that you can progressively upgrade.
It’s not just a cost for maintainers.
A better strategy for namespacing would go a long way.
The clojure.set thing is crummy but a hard call. That's a super fundamental piece of the library puzzle, and write once, run forever is a really very expected thing there. Those are something that a wrapper can overcome pretty simply though. In a lot of those cases the approach they take does seem reasonable though, namely that e.g. clojure.set/union perform exactly as is documented on the documented inputs.
LTS helps administrators and integrators maintain reliability, which is very important in the perception of one's (at least our) users. More important than keeping on the bleeding edge.
[1] At ng-conf 2017, the core team was asked whether they'd consider designating LTS releases, and the answer was essentially what you're saying here: They work very hard to be backward compatible, and encourage everyone to move _forward_ instead of getting stuck in the past.
From what I understand, they've done a better job from Angular 2-5.
Well, you can do it the Linux way, too: even if it is a bug, as long as fixing it breaks existing userland code, the fix gets reverted and the docs adjusted.
Compare this to anything except plain JS, jQuery and PHP - breakages all the time, everywhere, because no one values backward compatibility any more (python2=>3, for one, or whatever MS is choosing as the current framework to develop Windows applications now). This is the difference that MS took with the win32 layer and what made them successful - until they started ignoring BC in favor of loading new crap on consumers and enterprises alike every patchday.
Wait, people do that? If every PR lands on the next release and on the LTS release, how is the LTS branch different from the latest release? Isn't the point of LTS that it stays the same (except bugs) and isn't evolving?
I must be missing something obvious here.
I've worked at places that did stuff like the author suggests without pinning versions, and it always just ends up with some unlucky person spending their weekend fixing a dumb bug cause by an upgrade.
LTS is the reason we can all sleep soundly knowing our code won't spontaneously break, so I'm extremely grateful to the hard work everyone puts into it.
In my case, the only additional effort needed for node 4 is to use Array.prototype.indexOf instead of Array.prototype.includes, which is mildly annoying, and sometimes I forget and it causes a failure of the node 4 CI build, but really it's not that big of a deal. TypeScript handles the rest of the JS stuff, and the project doesn't use enough of the node API for those differences to matter. Other projects may different, but I think the right call is to drop support for old node versions when maintaining them is hard or because nobody is using them, not simply because they're old.
I want to use open source projects at work, not just in side projects, and at work we're on node 6 and always lag behind the latest node release by some amount, partly because upgrading node is a real project. We're a startup, not a massive enterprise, but it's still pragmatic to focus on real work instead of upgrading our dependencies all the time. If all of the open source projects out there only supported node 8, I think it would make them much less useful in the real world.
Object.defineProperty(Array.prototype, 'includes', { // eslint-disable-line no-extend-native
writable: true, configurable: true,
value: function (object) {
return this.indexOf(object) !== -1;
},
});
Array.prototype.includes = function (object) { // eslint-disable-line no-extend-native
return this.indexOf(object) !== -1;
}