Ask HN: Is the CI space overcrowded? | Hacker News Reader

Ask HN: Is the CI space overcrowded?

7 pointsScorpiion20 comments

Hosted CI is a hot space, and fairly recently the git hosting providers (Bitbucket and Gitlab) also entered the game with their own hosted services. Some hosting providers also have some overlap with CI functionality or specific CI services on their own. I wonder what people in the HN community thinks about the CI space as of today.

1. Do you think it's overcrowded? Is there space for new companies or is CI basically a "solved problem"? If not what do you dislike with current offerings?

2. Do you prefer a third party CI as compared to the hosting provider offering their own CI at a lower price? Assuming here that the hosting provider would offer lower price because their main income is from the 24/7 hosting fees. Or would you simply switch to the hosting providers CI if it was good?

Disclosure: I am the founder Sourcevoid, a cloud/PaaS company. I'm a little split on how much CI functionality we should integrate into our platform, compared to just trying to promote third party CI providers instead. We already do a fair amount via our build system, but at the same time CI is more than just the build process of course. This question is not about our company or our offerings, I just think it's an interesting question to discuss.

I find that it's just like the bug tracker space: There's a lot of contenders, but somehow none of them actually hits the sweet spot.

For CI, it's either: "too GUI" or "not enough programmable".

Also, CI is an area that absolutely cries out for container technology, but the state of containers on Linux[1] is absolutely abysmal. Maybe it'll be better in ~5 years when we'll hopefully be about 40% towards the capabilities with Solaris Zones. (Here's a hint: If you cannot fully 'contain' root and users/mounts/devices, then you're not being serious.)

[1] The most popular platform for this, by a far margin.

Which camp of "too GUI" or "not enough programmable" does jenkins fall in?

How would the sweet-spot look like?

"not enough programmable", but I'll add the caveat that I haven't really caught up with the 2.x bits that I think could help quite a bit.

(However, from the documentation I got the impression that the new 2.x Jenkins bits were quite repository-centric. It turns out that what I want is really to define the whole CI infrastructure separately from my repos with a few rules to generate appropriate jobs, etc. Perhaps the solution to that is to just define a repo for all-the-builds? Still need to look into it.)

I think currently you are supposed to use the "Multi-branch pipeline" for that, where you have separate branch in your repo for every configuration/job you would need.

We have decided to go different route, even though it is kinda held together by duct-tape :)

1. We have several shell-scripts that use the $JENKINS_URL/script to update global configuration. A.f.a.i.k. the /script endpoint gives you access to almost all of the jenkins internals to be operated with groovy. So thats how we set-up i.e. plugins, slave-providers, secrets, shared config-files, e.t.c

2. for job definitions, we use jenkins-job-builder [1] with the pipeline plugin [2] We then store both the job-builder configs and the jenkins-files they point to in a single repo, achieving the "define a repo for all-the-builds" solution.

3. to reduce repetition we used shared pipeline library, where we put all of (groovy) functions to be shared across jobs.

Neither shell-scripts for /script endpoint or the job-builder configs are particularly nice, but they get the job done. But we used them even before jenkins-files/pipelines/2.x

The 2.x bits do help quite a lot :-)

[1] https://docs.openstack.org/infra/jenkins-job-builder/ [2] https://github.com/rusty-dev/jenkins-job-builder-pipeline

Would be nice if we script those Jenkins pipelines with some other languages besides Apache Groovy. And why are Groovy's functional methods inhibited when used within Jenkins?

The reason why you can't use functional methods of Groovy in jenkins is, that the intepreter that runs inside of jenkins has been rebuild so that jenkins can suspend/resume the execution of the pipeline.

This means that everything needs to be serializable, and that in reality you are not really executing groovy statements, but rather passing continuations.

This is a reason why you have @NonCPS annotiation, that allows you to opt-out of that behavior.

FWIW at this point, this kind of demonstrates my point: I think I kind of want a library to build a CI system. Not some piece of software I have to run somewhere (as root, probably), which may enable me do to what I want.

Why hasn't anyone developed a CI-orcherstration library yet? Is it actually a difficult problem?

Is multi-branch pipeline really flexible enough to handle e.g Haskell/Cabal projects and Scala/sbt projects and "random things that I'll be happy to specify in a bash script?"?

Remember, I need this thing to be invoked by Gerrit, periodically, whenever there's a push (well, polling would be fine on this one), etc.

It turns out this is pretty complicated... and maybe nobody does it just right for you. (Hence my other comment. I want ultimate programability. As long as you give me visualization, then I'm OK. I can deal with having to edit a file and pushing rather than doing it through a GUI.)

I think an aspect of the problem is that most tools in this space actually get sold to project managers (and secondarily, process-oriented senior engineers), not hackers who just want to shut the door, quit Slack, and build stuff.

I think it's interesting to consider what hacker-oriented tools might look like. GitHub close to launch might qualify: simple interface, a model where (at least conceptually) projects are owned by individuals not organisations, and a simple but pretty pleasant-to-use issue tracker with no "workflow" type mechanisms. A lot of this has changed now (support for mandating code reviews!) and from a revenue point of view I'm not sure they're wrong to do these things. But it was the simple, hacker-friendly version that got them their initial mindshare.

I'm not entirely sure what the GitHub 1.0 of CI would look like, but probably not like the current offerings.

Nathan: You’ve shown with Freckle that you can enter a saturated market like time tracking and still do well.

Amy: “Saturation” is a load of bullshit :)

Nathan: Really it just shows the market exists.

Amy: It’s more than that. So much more than that. If a million people use Harvest, there’s no way they’re all served well by the same tool. The presence of other products doesn’t just show opportunity, it CREATES opportunity. Because wherever there’s a big biz, there will be lots of dissatisfied customers.

Source: https://stackingthebricks.com/difficulties-for-nathan-barrys...

If you could make a self-hosted Jenkins replacement that is even half as good as Jenkins (think extensions) but written in not Java, I think it'd receive a lot of positive reception.

The trick with CI, like task/project management software, is that people don't look at what works. They don't try and compete on intrinsics like stablility, extensibility, etc, and really after 12 months on any platform, any mid-large size company is going to care more about those things than new shiny.

Jenkins had the absolute worst UI for a long time, and it still was many people's top choice for CI. You're generally writing a tool for programmers, after all.

It's a well-populated space. If you're going to enter it, you need to be clear about differentiation.

We picked our CI solution because it was the only cloud offering we could find that supported Bitbucket and also allowed us to set up our own fully-custom environments to run against - without Docker.

Sadly, it took about a week's worth of research to figure that out. And I'm a bit concerned that in this crowded market they'll go out of business and leave us high and dry.

This sounds a lot like my requirements...can you tell me your CI provider?

https://buildkite.com

We migrated there from Bamboo Cloud when Atlassian discontinued it. It's nearly a direct drop-in replacement, and they even had docs specifically about Bamboo migrations. Their customer support during onboarding was also superb, and they've been responsive to feature requests. We've been using it for almost a year now, and overall we're pretty pleased.

What are you using currently?

I run an Azure VM hosted TeamCity/Octopus Deploy combo, with bitbucket for SCM.

Are you happy with that solution?

It's alright, I rarely have to touch it, but I'm no Sysadmin. If I had ops/sysadmin guys to shove onto, I would have no qualms using it long into the future.

I would, ideally, like a hosted build server, and a hosted Octopus - I really like it. Not the biggest fan of team city, but it works, and it's the only thing I know well.

Yeah, that's why we wanted a cloud solution - none of us really have significant sysadmin expertise, nor the time to devote to tending more boxes than we absolutely must. Only having to maintain the actual build environment has been a win.

We've had a couple occasions where we ran out of disk space (though not since we increased the disk size and added a cleanup job), and a few times a build agent has randomly stopped responding and needed to be restarted, but nothing we couldn't easily diagnose and fix.

I think that as evidenced by the discussion about concourse ci today https://news.ycombinator.com/item?id=14785254 that there is still room for improvement and new products, especially on-prem CI tools like Jenkins/TeamCity/Bamboo/Concourse.

One thing that I've always wanted was for Jenkins to get away from being a UI, and just be an engine with a nicely documented API. People could then build more niche CI/CD systems on top of that engine without having to reinvent the wheel. It's nice that Jenkins is finally getting a UI facelift though.

Reply on news.ycombinator.com