Hacker News Reader

Comment by nrclark | Hacker News Reader

Agreed with this sentiment, but with one minor modification: use a Makefile instead. Recipes are still chunks of shell, and they don’t need to produce or consume any files if you want to keep it all task-based. You get tab-completion, parallelism, a DAG, and the ability to start anywhere on the task graph that you want.

It’s possible to do all of this with a pure shell script, but then you’re probably reimplementing some or all of the list above.

View on HN

Just be aware of the "Makefile effect"[1] which can easily devolve into the Makefile also being "over there", far from the application, just because it's actually a patchwork of copy-paste targets stitched together.

[1] https://news.ycombinator.com/item?id=42663231

> use a Makefile instead

I was making a general comment that your build should be a single 'command'. Personally, I don't care what the command is, only that it should be a) one command, and b) 100% runnable on a dev box or a server. If you use make, you'll soon end up writing... shell scripts, so just use a shell script.

In an ideal world your topmost command would be a build tool:

     ./gradlew build
     bazel build //...
     make debug
     cmake --workflow --preset

Unfortunately, the second you do that ^^^, someone edits your CI/CD to add a step before the build starts. It's what people do :(

All the cruft that ends up *in CI config*, should be under version control, and inside your single command, so you can debug locally.

That's exactly why the "main" should be shell, not make (see my sibling reply). So when someone needs to add that step, it becomes:

    #!/bin/sh

    step-I-added-to-shell-rather-than-CI-yaml
    make debug  # or cmake, bazel

This is better so you can run the whole thing locally, and on different CI providers

In general, a CI is not a DAG, and not completely parallel -- but it often contains DAGs

Make is not a general purpose parallel DAG engine. It works well enough for small C projects and similar, but for problems of even medium complexity, it falls down HARD

Many years ago, I wrote 3 makefiles from scratch as an exploration of this (and I still use them). I described the issues here: https://lobste.rs/s/yd7mzj/developing_our_position_on_ai#c_s...

---

The better style is in a sibling reply -- invoke Make from shell, WHEN you have a problem that fits Make.

That is, the "main" should be shell, not Make. (And it's easy to write a dispatcher to different shell functions, with "$@", sometimes called a "task file" )

In general, a project's CI does not fit entirely into Make. For example, the CI for https://oils.pub/ is 4K lines of shell, and minimal YAML (portable to Github Actions and sourcehut).

https://oils.pub/release/latest/pub/metrics.wwz/line-counts/...

It invokes Make in a couple places, but I plan to get rid of all the Make in favor of Python/Ninja.

Portability to other CI/CDs systems is an understated reason to use a single build command.

You invoke CMake/qmake/configure/whatever from the bash script.

I hate committing makefiles directly if it can be helped.

You can still call make in the script after generating the makefile, and even pass the make target as an argument to the bash script if you want. That being said, if you’re passing more than 2-3 arguments to the build.sh you’re probably doing it wrong.

Yes to calling CMake/etc. No to checking in generated Makefiles. But for your top-level “thing that calls CMake”, try writing a Makefile instead of a shell script. You’ll be surprised at how powerful it is. Make is a dark horse.

I wouldn't be surprised at all, make is great!

My contention is that a build script should ideally be:

sha-bang

clone && cd $cloned_folder

${generate_makefile_with_tool}

make $1

Anything much longer than that can (and usually will) quickly spiral out of control.

Make is great. Unless you're code-golfing, your makefile will be longer than a few lines and a bunch of well-intentioned-gremlins will pop in and bugger the whole thing up. Just seen it too many times.

Edit: in the jenkins case, in a jenkins build shell the clone happens outside build.sh:

(in jenkins shell):

clone && cd clone ./build.sh $(0-1 args)

(inside build.sh): $(generate_makefile_with_tool) make $1

I have experienced horror build systems where the Makefile delegates to a shell script which then delegates to some sub-module Makefile, which then delegates to a shell script...

The problem is that shell commands are very painful to specify in a Makefile with weird syntactical rules. Esp when you need them to run in one shell - a lot of horror quoting needed.

Reply on news.ycombinator.com