Hacker News Reader

Comment by chockchocschoir | Hacker News Reader

How do you as a solo developer get blocked on anything related to GitHub? Are you doing all project planning on GitHub without any other copies elsewhere? Most if not everything you do on GitHub could be replicated locally, one way or another.

I can understand the problem for larger development teams, as a lot of communication and workflows can happen via GitHub Pull Requests or similar.

View on HN

Personally, I can't deploy if GitHub is down. CI pulls from GitHub. One of my projects includes libraries as submodules, with GitHub as the origin. The deployment configuration has everything cloning from GitHub.

> Most if not everything you do on GitHub could be replicated locally, one way or another

That's not really the point though.. If you have services in github etc, why would you replicate all of them locally too for unplanned downtime ?

Well, usually you figure out how to run your tests, check code coverage and build your binaries locally first, then figuring out how to get it to work in the CI environment, not the other way around. So the replication has already been done, but in the direction of local environment > remote environment.

I don't disagree, really local environments should be able to manage dev and most daily tasks, but large integration pipelines, test automations etc.

I just don't feel the need to defend GitHub here for the reason of : "Well you should be able to manage locally anyway"

This is quite similar to the somewhat philosophical question of having backups: if your job is to make sure your company has backups of their data, and you entrust a single vendor with that data because they take care of backups so you don't have to, are you really doing your job? Said another way: if you (or your immediate team) are not the party that turns one thing into two things, then there is still a SPOF in your wheelhouse, and the whole point is for that to not be the case.

Maintaining an entire secondary deployment and CI system just seems unrealistic to me. I'd get so much flack about the extra spend and when the systems get out of sync and causes downtime, that investment ends up with net negative for all involved.

SPOF is something that you have to make trade-offs for and decide what you can cope with.. How much work to maintain a parallel infrastructure _just in case_ GitHub goes down for more than an hour once or twice a month? How many people actually are in the position to pivot their entire company processes around this failure case?

It can't be "remove all SPOF or you aren't doing your job". How many offices are resilient to continue working if the building power goes out?

Since most executives dont actualky care if backups work for real just that they are not liable if the backups fail. Yes.

I am a solo dev, but I work in a company and now have been working on pushing something to test it on Github actions. If it was something different, maybe. But I understand how I might have miscommunicated that.

I'm also in a company, but today am building up something on Github Actions.

Every single time I start doing development on Actions, GitHub goes down or actions start failing. I'm not sure if this is saying something about me, or GitHub.

Try building any Rust project while GitHub is down.

That's only applicable if you use crates directly from GitHub repositories right? I've been sitting and developing+building a Rust project for the last +10 hours (except the minutes when I was here arguing on HN) and never hit any issues, but I'm using crates either directly from disk (cloned from GitHub initially) or from crates.io.

I could be wrong, but try a fresh build with a clean CARGO_HOME (or maybe just a cargo update?). It seems to query a lot if you don't have a full cache.

While it's true that Cargo will attempt to check on some things, you can pass flags to tell it to not bother. Specifically, https://doc.rust-lang.org/stable/cargo/commands/cargo.html#o...

Maybe using their CI/CD?

CI/CD should you perform tasks that you could execute on any machine, besides the deployment, which should work differently locally VS via CI service. But it should still be possible to deploy without CI, otherwise you set yourself up for being unable to deploy when you really need it.

Yeah, but theory and practice are very different things. If CI always worked for me and my experience was great with it. I will not invest time to do independent CI for everything. Hence depending on Github Actions. Clearly not the best approach, but it's about being pragmatic also.

What do you mean "independent CI"? You start with a `Makefile` with the commands for testing, building and so on, and CI is just calling those commands but on another instances, that likely gets triggered for each commit/tag/pull request/whatever.

When you first setup CI, you don't actually just develop for the CI environment first, that's the second step after you know how to perform the same things locally. So not sure why there would be "independent CI for everything".

It's easy enough to unintentionally box yourself into a hard 3rd party dependency. Like the github secrets store, conditional logic or triggers in the ci.yml files, pipeline driven semantic versioning, etc.

Github-actions is not created to encourage that though. It uses syntax that only works on Github. Look at their workflow-examples and you will see.

Don't you define things in your repository first (usually via a `Makefile`) and then call those things in the CI environment? Or are you building things differently in the CI environment compared to the local environment?

I mean, if you're working on a project that has tests, code coverage and binary builds setup, you usually have a `Makefile` or `package.json` or whatever to run your scripts, and your CI setup just calls those very same scripts but in their environment (sometimes with different arguments/environment variables).

Not sure why it would be different for GitHub Actions. It's certainly how I use it day-to-day.

no that's not how it works at all. The "actions" are proprietary to GitHub and hosted on GitHub. People create custom actions and allow others to reuse them. Everything is hooked in to GitHub via their proprietary yaml config.

> Not sure why it would be different for GitHub Actions.

because vendor lock-in. GitHub doesn't want to make it easy for you to switch.

You can run github actions offline

https://github.com/nektos/act

I don't think github is trying to create lock-in, I think rather they were trying to make a way to easily share actions (not sure what other CIs systems are designed to have an ecosystem of publicly shared actions). The actions are public and therefore easy to make something that interprets them.

I can only guess at some point there will be a push for CIs to converge on some "actions" standard, maybe?

That's neat, though it is a 3rd party trying to replicate it.

Gitlab's open source runner supports parsing the ci yml and running a job locally, presumably using the same code as the platform, like:

  gitlab-runner exec docker some-job-name

Though they would both suffer on any dependencies on platform hosted environment vars, secrets managers, etc.

Are there any CI/CD metasystems that aren't vendor locked-in? GitHub/even GitLab, Jenkins (freestyle and scripted)/Azure/Travis.... all vendor-specific, as far as I know.

Sure, have all the heavyweight stuff in separate scripts that are just called, but platform specification/multiple platform builds/specifics of caching/secret handling/deployment handling are always different. Some tools (e.g. codecov) do abstract over some platforms, but not all, and the GitHub Actions model of "here is a literally pre-prepared step in your pipeline" can be pretty appealing.

It's literally, pick your poison, and resign yourself to reimplementation if you ever need to switch platforms.

If you have vendor lockin with GitHub Actions, it's because you chose to do it to yourself. Nothing prevents you from using only the `run` action to run a shell script so that everything that CI does can also be done on your dev machine.

Both my personal projects and my $dayjob repositories have every test, etc triggered via `make test` or `test.sh`, then the GitHub Actions workflow YAML just `run`s it. Secrets also work fine - the makefile / shell script expects them to be defined as env vars, so the developer running them locally just needs to define those env vars regardless of how they obtained the secrets.

https://news.ycombinator.com/item?id=25104253

Sure, but then I guess you always install the dependencies from scratch, can't utilize cache, and won't have a dependency graph as well.

Parallelising your build/deployment will likely also be harder to do.

>Sure, but then I guess you always install the dependencies from scratch

Sure? On your dev machines the dependencies are already installed. On GHA VMs the network is fast enough that installing deps is not slow. What's the problem?

And if you really have a problem, presumably you have some master tarball / container image that your devs use to set up their dev machines because installing your deps is so complicated, so scp / pull that in your script?

>can't utilize cache

A cache is necessary for CI VMs that are cleaned for every run. When building locally your dev machine already has everything the cache would have.

>and won't have a dependency graph as well. > >Parallelising your build/deployment will likely also be harder to do.

You realize the two things shell scripts are good at is running commands either series or parallel, exactly how you want them? Instead of learning a brand new DSL to be able to do `if` and `&`, you just write `if` and `&`. This is already covered in the comment I linked.

You make some good points and I tend to agree with your approach. What you have to do though is skip using "uses: actions/", "needs", and won't be able to use other actions that has been published to the Github marketplace. So you have to make a conscious decision to go against the way people usually uses Github actions and won't be able to utilize some of their features. You avoid vendor lock-in though, which is what we are talking about :)

Right, that's the idea. Not only are these "actions" the cause of lock-ins, you also have to treat them as dependencies that you're trusting with your repo integrity, secrets, etc. As soon as you start using third-party actions you have to start worrying about https://docs.github.com/en/actions/security-guides/security-...

SourceHut is one source code hosting with the right idea here. Its CI only has the equivalent of the `run` command for running shell scripts - https://man.sr.ht/builds.sr.ht/manifest.md#tasks

Yes. I have debated with myself before which route I should take. I went with the Github way, just because I felt uncomfortable making a decision like this that goes against all the examples. Perhaps it was the wrong approach. Another problem with the Github way, is that functions/code-reuseability is almost non-existent.

thanks for the downvotes.

> it's because you chose to do it to yourself.

I didn't choose shit. My company did. Why are you putting this on me?

> Both my personal projects and my $dayjob repositories have every test

Congrats on not actually using GitHub actions? I guess?

So many people here sucking Microsoft cock. And there is yet another incident today! What's that make, three days in a row now? Four, if we're actually counting. They aren't even hitting 2 nines uptime. Two. Fucking. Nines. Going on many years now. But apparently that is just fine because everyone is running self-hosted infra in parallel to their cloud shit.

>Congrats on not actually using GitHub actions? I guess?

>But apparently that is just fine because everyone is running self-hosted infra in parallel to their cloud shit.

In your haste to complain about downvotes and accuse other people of "sucking Microsoft cock", you forgot to actually read the comment you replied to.

Things in the comment you replied to:

1. An assertion that one can write their CI in a script that is not tied to one CI vendor, so that it's easy to run the same steps as what the CI does locally or in another CI. ie, no lock-in to the current CI.

Things not in the comment you replied to:

1. An assertion that I run self-hosted CI in addition to GitHub Actions.

2. An assertion that GitHub has good uptime.

In my case, at least, GH actions is the only place with all the secrets necessary to deploy my (small) webapp. Sure, I can generate alternative tokens and pull some things out of 1password, but it'd be time consuming. (Also, changing things like JWT secrets is less than ideal.)

There's also just the number of things it checks. jest runs, lint/build, e2e and acceptance tests, 2 docker builds pushed into ghcr, and then ansible to deploy. It's mildly error-prone to do myself, especially the docker and ansible steps because that's where the secrets come in.

So sure, it CAN be done manually, but the entire point of CI/CD is to do everything consistently, repeatedly, and without the risk of manual error. It took me hours to figure things out the first time. Why would I want to risk doing things manually now?

> But it should still be possible to deploy without CI,

Solo? Yes. In a team? Less often just because you might have credentials stored in secrets that are not shared with the entire team for security reasons.

In larger corps it's easier to wait it out that go through the trouble of escalating permission requests.

But I do get your point. Personally for deployments I have Ansible playbooks that are invoked via straighforward calls which can be done locally.

Reply on news.ycombinator.com