Hacker News Reader

Comment by ltbarcly3 | Hacker News Reader

Ok, if I understand this properly:

They have a core library that other projects use.

They want to release a new version of this core library, and upgrade all the projects to use the newer version.

If the CI breaks for the projects using the library, they want to catch it.

This still depends on the projects using the library having enough and good enough testing for something to break in the CI when this core library does something bad. That is a huge investment (having those high quality tests). The rest of this is just a script that runs the tests from those projects when people merge code into the core library, which seems like very much not a big deal.

View on HN

The former approach was release + upgrade all projects. Now if one of those fails post-update you have to build a new release and upgrade all projects _again_. Multiply that by hundreds of services and it can become pretty annoying.

By pulling all the reverse dependencies and testing those _before_ releasing & upgrading you reduce the chance that you need another bugfix release.

And with hundreds of projects the average quality of tests does not matter that much. Some projects will happily pass, while some may fail. Those that fail will provide valuable insights.

> The former approach was release + upgrade all projects. Now if one of those fails post-update you have to build a new release and upgrade all projects _again_. Multiply that by hundreds of services and it can become pretty annoying.

I think there are actually at least 3 possibilities when you find a new version of a component breaks something that depends on it?

  1) Continue using the old version for however long it still meets it's requirements and then do (2).

  2) Make the local changes required to be compatible with the new version.

  3) There is actually a bug in the new version and you need to fix the dependency.

If (3) is happening a lot it suggests that your test suite isn't good enough or you have some design problems that are making it impossible to provide a stable API.

What it sounds like is that they don't have many unit tests for jvmkit. So rather than write extensive unit tests, they're using integration testing with all the dependencies. I guess it makes sense, network stacks are hard to unit test.

Yes, and the whole solution allows you to plan / decide on (2) and (3) - both need that awareness.

And (1) is common in the open source and cross organization world. But it is less desirable if both the consumers and producers of the library work in the same place. Version spread causes maintenance cost. And at one point it was a common theme that "an upgrade of JVMKit would have avoided this incident".

Reply on news.ycombinator.com