Hacker News Reader

Comment by cocoflunchy | Hacker News Reader

I'm considering using Hetzner (or equivalent) for our CI setup with Github Actions custom runners. Right now we're on CircleCI and it costs an arm and a leg.

Does anyone have experience to share with that kind of setup? What's the maintenance like?

View on HN

I'm using Hetzner for our CI setup with GitLab-runners. With a simple cloud.init script, it's cheap and I have nearly 0 maintenance: https://gitlab.com/21analytics/gitlab-runner-cloud-init

Same, only maintenance is that it runs out of disk space every once in a while. Have a cron script that does various docker pruning tasks now, so it's become fewer and farther in between very time I discover a new leak.

I had the same issue so I made the cloud-init script create a systemd service that does the docker cleaning.

That's exactly one of my use cases, I have setup runner for 2 projects last week and action runtime went down by a large margin, from 10-15 to 3-4 minutes.

I use single dedicated server that costs ~40EUR/month, AX41-NVME, and each runner is a separate user account to allow for some isolation.

Depending on your setup, you might need to spent some time adjusting jobs to have proper setup/cleanup and isolation between them (but it's not really Hetzner specific, just general issue).

This was one of my Q4 projects at work last year. We moved CI to 3x hetzner machines, each running four copies of the self-hosted github runner, and drove our build/test times from >20min down to 3-4 min on average. It's ridiculous how big a difference running on a capable bare metal box makes. We run a thousand or so builds daily and pay about 300 euro a month for the setup; our overage fees from github actions were often higher than that. Reliability has been "ok": one of the machines started throwing errors that smell like bad RAM/CPU (bus errors, random reboots, etc), we raised a support ticket, they nuked it and gave us a fresh one.

We provision them with ~200 lines of shell script, which we get away with because they are not running a "prod" workload. Don't forget to run "docker system prune" on a timer! Overall these machines have been mostly unobtrusive and reliable, and the engineers greatly appreciate the order of magnitude reduction in github actions time. I've also noticed that they are writing more automation tooling now since budget anxiety is no longer a factor and the infrastructure is so much faster.

Short answer it depends on your needs and you comfort level with server management. But generally I would say a server running something like Jenkins or similar is not that taxing on the maintenance budget. It does have some up-front cost in getting everything configured and running as you like but after that it is fairly easy to maintain. I usually automate the provisioning as much as possible both for the self-documenting aspect but also to make it easier to repave the system or spin up additional nodes as needed.

I’m running a kubernetes cluster on Hetzner with the GitHub actions runner chart. Zero issues and much faster (and cheaper!!) than GitHub’s runners.

My only issue is that security scanners can’t run on self-hosted runners (GitHub refuses the artifact result, so technically, they do run, but the results fail to upload).

> or equivalent

Do you have any alternatives? I thought Hetzner was fairly unique in their dedicated server offerings (for the price, I mean).

OVH (with their low-end brands such as SoYouStart and Kimsufi) are in the same ballpark.

Reply on news.ycombinator.com