Hacker News Reader

Comment by danpalmer | Hacker News Reader

I've used GitHub Actions quite extensively now, across infrastructure automation, Python CI/CD, and iOS CI/CD, and while not perfect, it's the best platform I've used for this stuff so far.

Compared to Jenkins it needed far less maintenance. Compared to CircleCI it felt much easier to work with and to build reliable pipelines due to the locking primitives it provides, and compared to Semaphore I found it easier to understand how the pieces fit together.

My criticism would mostly be about missing features, but the pace of development has been great and the only one I have left on my list is SSH debugging, which didn't end up being much of a blocker to our adoption anyway.

As for reliability, nothing is perfect, but in my experience it's at-or-above the level that CircleCI provided, and far surpassed our in-house Jenkins server.

View on HN

We use GitLab CI for internal code and GitHub Actions for public code, and are generally happy with both.

GitHub Actions emphasizes reusability and composability around “actions” as installable packages suitable for a “marketplace” model. This works well, but once you deviate from a standard action, changing its behavior requires learning (or re-learning every six months) the GitHub Actions DSL to make it do what you want. This can be frustrating when it feels like you need to model your task to fit within an abstraction that didn’t help you in the first place.

GitLab CI feels much more “raw” and closer to the metal – there is less emphasis on packaging code into reusable workflows, and more emphasis on the practicalities of operating robust CI pipelines. Yes, you could have a different runner for every task, and “package” CI code into Docker images similarly to how GitHub Actions packages it into workflows. But the service isn’t designed around that idea. And for internal pipelines, that feels like a needless abstraction. It’s nice to keep pipelines simple and robust.

Put another way: there is much less “magic” in GitLab CI than in GitHub Actions. In many ways GitHub Actions feels more like a toy than a tool. This is generally a good thing, IMO, and overall I’d say the tradeoffs are worth it for both services, especially given the different contexts (internal vs. public code) in which we use them.

I forgot to mention GitLab CI, but in my experience, much like with Travis CI, it is as you say lower level. I do not find that to be a good thing.

Rather than lower level being faster as it is with code, generally with CI we want to raise the abstraction level so that it's easier to make use of things that make CI run faster – parallelism in workflows, caching etc.

As for reliability, I also found GitLab CI lacked the concurrency control caching primitives necessary to trust continuous delivery pipelines.

I found you could either write a slow correct pipeline, or a faster but fundamentally unsafe one.

Thanks for the feedback! I'm one of the PMs for GitHub Actions, and I appreciate this. Thinking about Actions as a set of primitives that you can compose is very much how I think about the product (and I think the other PMs as well) so I'm glad that resonates.

We're always welcome to feedback, and we're continuing to invest and improve on the product, so I'm hopeful that we can address the features that you're missing.

Here's my ask:

* Setting up GHA is still a lot of "commit and hope for the best". I've resorted to having a sandbox repo just for experimentation/testing so that I don't overly pollute repos that I actually care about. It would be great to get more instrumentation to see what is going on.

* I have a monorepo for Dockerfiles. It's quite annoying that I have to have separate invocations for different Dockerfiles in dependabot.yml. I should be able to specify /Dockerfile or /Dockerfile* as patterns for detection. The Dependabot invocation for GitHub Actions is a single entry and it would be great to have that.

* I quite like Step Security's Harden Runner but it does require more work/invocations to get this set up. Maybe GH can work with them to more closely incorporate said functionality?

* Make the cache bigger? I build a fair number of multi-arch containers and starting all of them at once tends to blow out the cache.

* Given the interest around sigstore and SBOMs, maybe incorporate native capabilities to sign artifacts and generate SBOMS?

> Setting up GHA is still a lot of "commit and hope for the best". I've resorted to having a sandbox repo just for experimentation/testing so that I don't overly pollute repos that I actually care about. It would be great to get more instrumentation to see what is going on.

There is act[0] which aims to let you run github actions locally via Docker. It isn't perfect but it does a decent job at it, and for the most part your pipeline can be run locally.

After MS bought GH, I had hopes that they would build a tool to run action locally, but nothing yet.

[0] https://github.com/nektos/act

I've had no luck reproducing problems in Actions with act, and the rest of the time have problems in act that I don't in Actions it seems.

I like the idea and also would like something first-party, but I imagine it's hard and GitHub would want it to be less buggy than act is, and maybe they're trying but it's not there.

Tbh even if it ran remotely in actual Actions, but just didn't show up in the repo UI, logged locally, that would be fine?

Thanks. The "commit and hope for the best" problem really resonates with me. There are two great projects that might provide some pain relief - nektos/act or rhysd/actionlint. But I agree that commit-to-validate is probably the best strategy at the moment, which is deeply unfortunate. This is an area that I intend to improve in the future.

As for the cache, we doubled it at the end of last year to 10GB. https://github.blog/changelog/2021-11-23-github-actions-cach..., but I can see how multi-arch images would be very large. Have you considered putting images into GitHub Container Registry instead of putting the layers into the cache? I'd love to understand if that is appropriate for your workflow, and if not, what the limitation there is.

Appreciate the rest of the feedback, I'll pass it along to the appropriate teams.

From my perspectives, GHA is missing 2 things over CircleCI. A way to pause an action for approval, or a way to pull artifacts from other workflows. Both of these actions are _possible_ with an external service but painful to setup. I want to: create a terraform plan, approve it, and then deploy the specifically approved plan. That's not so difficult in CircleCI but is _painful_ in GHA.

fwiw, you can use approvals using environments: https://docs.github.com/en/actions/deployment/targeting-diff...

Sure, but that's actually worse than useless for my use-case. Image this, you have an action that publishes your plan to your PR (#1 - it's a biggish feature). It gets merged and goes to approval. Then people happen. PR #2 is addressing a customer-facing bug so it gets fast-tracked and rammed through before PR #1. Suddenly PR #1 is silently invalid. It _should_ be rejected at this point but the whole point of CI/CD is to save time and reduce the surface area for human mistakes.

specifically for your terraform example, wouldn't it make more sense to have the PR merged only when apply was successful?

i'm not sure how well that can be represented in GH actions, but that would surely be the better option?

you'll always risk some kind of race condition there, e.g. atlantis locks the project while something is planned but not applied to avoid such things from happening. this of course prevents having multiple PRs "ready" at the same time, you'd have to unlock the active PR lock to be able to implement another one.

This still can't use GHA to enforce any sort of integrity so it's kinda moot. I have some of my projects set up to deploy with CircleCI...which can give me the build, approve, apply (specifically the thing you approved) chain that I'm looking for (so there's no race condition). "Why not use CircleCI?" well i do, but if my company decides to cut costs, it may not survive the chopping block...so I'm looking at other options.

Just looked at CircleCI recently. Great product, but the price difference between them and GHA was absolutely jaw dropping.

That's very much how I think about it too, which is why it frustrates me that I can't create canned workflows that apply to all my repos of a certain type (language specific linting and releasing say).

I know I can create user/organisation templates, but all that does is put it in the UI chooser to create a commit to put it in the repo from the web. I want to do something like `include: OJFord/workflows/terraform-provider.yml` or `include: OJFord/workflows/rust.yml`

Perhaps even better would be I don't even have to specify that in the repo, they just apply automatically to any which match a given pattern - named `terraform-provider-*` or having a file `Cargo.toml` say - but I realise that's probably too big a deviation from the way Actions works at this point.

Could you stuff your actions definition into a sub module pointed at trunk?

Interesting idea! Even if Actions will follow a submodule though (I doubt it tbh, it happens before any actions run of course so we have no control over that) you can't point it 'at trunk' as far as I know, they're always at a specific commit.

(E.g. if you git submodule foreach git checkout master, your diff if you have one will be updating commit hashes, not -whatever +master. This is good for a lot of other reasons but doesn't help here.)

This might be what you're looking for: https://github.com/github/roadmap/issues/74

It's definitely more pleasant to use than Jenkins, but Jenkins is as painful as it gets.

90% of the times it's some ancient setup that nobody knows how it works. And the amount of hands-on required to keep in working is waaaay higher than other CI pipelines.

Circle CI is also not very nice at all. Things like GitLab CI or Sourcehut Builds are sooo much simpler, lighter and faster in comparison.

Jenkins sucks from a UI perspective, and from a maintenance perspective, but it's very powerful in ways that most other CIs are not.

The classic example for us was concurrency control. We did continuous delivery, and so it's critical that you're not trying to release the `master` branch multiple times concurrently, especially when database migrations might happen, etc.

With Jenkins you could quite easily say "only 1 of this job at a time", but this wasn't (at the time) a feature of CircleCI, GitLab, or Travis. GitHub Actions added it about 2 years ago I think, and Semaphore has had something similar for a while.

While I agree Jenkins is painful, having this feature and the reliability it brought reduced pain significantly. CircleCI not having this feature, for us, caused a lot of pain.

> My criticism would mostly be about missing features,

My only real requested feature is one that's been open since early 2020 (this issue replaced the old 2020 issue) and has the status "Status: Q1 2022 – Jan-Mar" https://github.com/github/roadmap/issues/161

Without this, i'm relying on Azure actions to start/stop+deallocate a self-hosted runner, which adds ~1 minute to the job startup time.

> SSH debugging

Have you seen https://github.com/marketplace/actions/debugging-with-ssh? (Maybe this only works with public Github - that's all I use). I add an `if: ${{ failure() }}` to only spawn the SSH when my job fails and use that to debug CI issues.

Reply on news.ycombinator.com