Hacker News Reader

I work on a project with 4.4 million lines of code and using a single Makefile with no generated code works fine. It's really not all that difficult.

Projects of much smaller sizes often have recursive convoluted makefiles.

> Projects of much smaller sizes often have recursive convoluted makefiles.

You name any technology and anyone can enumerate dozens of projects that use it wrong.

I’d walk that before talking. Take any complex makefile system and turn it into a single “not really difficult” makefile without sacrificing anything important. Wins this argument and helps those who “use it wrong”.

And I can show you thousands of "Hello World"s that use GNU Autotools or CMake ;)

But seriously: can I take a look at it (Soource + Makefile)?

This is most likely what is being referenced: https://github.com/jart/cosmopolitan/blob/master/Makefile

I like how the includes are separated and commented.

Also if you weren't already familiar with their work you might be interested in giving this a read: https://justine.lol/ape.html

Can

  % grep '^include' Makefile -c 
  159

includes with

  % wc --lines --total=only $(awk '/^include/{ print $2 }' Makefile) Makefile 
  22547

lines in total really count as having a single makefile? I have no dog in this fight, just wondering.

I don’t object to “it works for me”, but “it’s really not all that difficult” is a bad generalization.

* If you need portability, Makefiles are hard.

* The whitespace design of Makefiles is bad and has swallowed up countless debugging hours over the years. This design flaw isn’t intrinsic to the way Makefiles work, it’s just a lousy artifact from a superficial decision from decades ago: to change behavior based on distinctions invisible in source code. It’s mitigated by syntax highlighting but still bites people.

* Makefiles are dependent on the consistency of the build environment, for example the availability and behavior of command line switches. Even if your project doesn’t need OS platform portability, this is still a pain across time and requires external tooling to manage.

* There are certain subtleties to the way Makefiles behave that are addressed by `.PHONY`. I agree that these are manageable in the absence of other complexities, but they contribute towards Makefiles being more difficult than appears at first.

I’m sure you’re familiar with those critiques and others. They may not bother you, but you don’t speak for everybody.

My Makefile is portable. It builds binaries that run on six OSes and two architectures. So I used my Makefile to build GNU Make and a GCC toolchain. Now I can run my Makefile on any of those OSes / architectures too, and it'll produce the same deterministic output, bit for bit.

If part of your build is building your own build tool in order to ensure you have the proper build tool then why not build a different “better” build tool?

Part of the premise of Make is its ubiquity, but if you can’t rely on that save as a simple bootstrap (as you seem to be doing) then why not forego it for something else?

Are you suggesting this: https://xkcd.com/927/ ?

> (..) then why not forego it for something else?

Because blindly ditching a technology for no reason at all is not a way to fix problems.

My project is literally to make a compiler toolchain. Do you expect me to not use it? The nice thing is that you can build my project on any computer that has modern GNU Make. That and sh are the only things that needs to be installed.

> My Makefile is portable.

Oh yes, in the good old tradition of "... as long as it's some Linux on x86".

    [...] on Linux 2.6+ (or WSL) using GNU Make.

Sorry, it's actually AMD64 _and_ ARM64!

People use Windows toolchains and nobody cares. Those are significantly less portable. But here I am, building with Visual Studio.

You might want to take a look at the "actually portable executable"* project, made by the person you're responding to. There may be tips that will make make more approchable to you, if you're still dealing with MAKEFILEs.

* https://justine.lol/ape.html

Sure buddy anything can be manageable once you invest enough time and sanity.

Now show us the Makefile.

> I work on a project with 4.4 million lines of code [...] It's really not all that difficult.

You may be biased.

I don't understand this statement, "which hasn't been part of POSIX up to the 2017 version - IEEE Std 1003.1-2017."

I've definitely been using .PHONY on various Linux and MacOS computers long before 2017.

Maybe it's just me, but I've never much cared for whether or not something is specified if it happens to be present everywhere I go.

> I've definitely been using .PHONY on various Linux and MacOS computers long before 2017.

Me too, and I've also used Makes which didn't (on e.g. Irix). What I wanted to express had been that you can't even rely on `.PHONY` existing, much less many other features.

I think it's pretty reasonable to expect GNU make to be available, and rely on its features. IRIX is dead for 18 years now.

If I may cite myself:

    Don't forget that not every Make is GNU Make, BSD and other nix like Solaris/Illumos still exist.

And I'm not even talking about Windows and Nmake.

Both BSDs and Solaris provide gmake via official channels.

There is "make" for Windows too, but it's not as relevant - the windows commands are so different, it's unlikely you'll have a cross-platform file for Windows and BSD/Linux, unless you require user to install unix tools on Windows, in which case they will likely come with GNU make.

I think in most cases, using GNU make is the easiest way to provide compatibility with multiple OSes. There are certainly exception - if your daily driver is FreeBSD, use BSD make. But for Mac OS or Linux users, GNU make is a good default.

> Make does exactly one thing: it takes input files, some dependencies and generates _exactly_one_ output file.

Not true. Your dependency graph might culminate on a single final target, but nothing prevents you from adding as many targets that generate as many output files as you feel like adding and set them as dependencies of your final target.

Think about it for a second. If Make was only able to output a single file, how in the world do you think it's used extensively to compile all source files of a project, generate multiple libraries, link all libraries, generate executables, and even output installers and push them to a remote repository?

> To have rules which don't generate output (like `install` or `all` or `clean` or all targets in the article) we need to resort to a hack, a special magic target like `.PHONY`

I don't understand what point you thought you were making. So a feature that boils down to syntactic sugar was added many years ago. So what? As you showed some gross misconceptions on what the tool does and how to use it, this point seems terribly odd.

> And the problem isn't that they use it, but that they are proposing it as a solution to "everybody".

I think you're making stuff up. No one wants Make to rule the world. I don't know where you got that from.

I think the whole point is that Make excels at a very specific usecase: implement workflows comprised of interdependent steps that can be resumed and incrementally updated. Being oblivious of Make leads many among us to reinvent the wheel poorly, using scripting languages to do much of the same thing but requiring far more work. If you can do this with a dozen lines of code in a Makefile, why on earth would you be churning out hundreds of lines of any random scripting language?

> Not true. Your dependency graph might culminate on a single final target, but nothing prevents you from adding as many targets that generate as many output files as you feel like adding and set them as dependencies of your final target.

Sorry, I did phrase that badly. A better version of that sentence would be

   A single target (a single node in the dependency graph) of Make does exactly one thing: it takes input files, some dependencies and generates _exactly_one_ output file.

> I think the whole point is that Make excels at a very specific usecase [..]

Excatly what I wanted to express with my post above. But the article isn't about such a case, but for something for which a single shell script (or, better, just adding the commands to the `scripts` stanza of `package.json`, which is the more common, expected way to do it) is actually better suited and way less error prone.

A single target (a single node in the dependency graph) of Make does exactly one thing: it takes input files, some dependencies and generates _exactly_one_ output file.

Yes, but this is not particularly relevant to the user. With pattern rules it's trivial to define a large number of targets automatically, such as in the example (from the manual):

    objects = foo.o bar.o

    all: $(objects)

    $(objects): %.o: %.c
            $(CC) -c $(CFLAGS) $< -o $@

I have been unclear in my formulation, sorry.

The problem is a target with more than one output files, that target would look something like, which does work

    foo bar: baz
         compile $< -o foo -o bar

but as this is the same as writing

    foo: baz
         compile $< -o foo -o bar

    bar: baz
         compile $< -o foo -o bar

to generate `foo` and `bar` the rule is run twice in a parallel build (`make -j 2`. Which may just be unnecessary or it may break the whole build.

So can that be solved like this, or is there some other subtlety I'm missing?

  foo: baz
    compile $< -o foo -o bar

  bar: foo

(Can someone tell me how to do code tags on HN please? :) Edit: fixed now, thanks!)

> Text after a blank line that is indented by two or more spaces is reproduced verbatim. (This is intended for code.)

(from https://news.ycombinator.com/formatdoc)

Among other things, now you have to maintain a set of dummy targets. If you have a variable (possibly generated) that is basically

  A_MESS_OF_FILES := foo bar zot

You now have to create dummy targets for bar, baz, and zot and not forget to add them. Or maybe break it into

  MAIN_FILE := foo
  SUBORDINATE_FILES := bar zot

  foo: baz
    sudo make me a foo

  $(SUBORDINATE_FILES): %: foo

Subtlety. Now the dependencies are `baz` -> `foo` -> `bar`, that is `foo` is a temporary target and won't be (re)build if `bar` already exists. Which may or may not be a problem. This temporary target (whatever the actual term is) can be "elevated" to a "normal" target with the use of some special target (which I'm too lazy to look up right now).

https://www.gnu.org/software/make/manual/html_node/Multiple-...

Scroll down to "Grouped targets". I think these address the concern you raise.

The snippet you have shown is an example of the "independent targets" pattern which the first half of that page also covers.

Thanks, that's it. It has been introduced with GNU Make 4.3, January 2020 https://lists.gnu.org/archive/html/info-gnu/2020-01/msg00004...

> A single target (a single node in the dependency graph) of Make does exactly one thing: it takes input files, some dependencies and generates _exactly_one_ output file.

I'm still not really following the point about one output file? That might be Make's stated purpose, but a Makefile rule can certainly create extra files as a side effect (or do pretty much anything a shell user could do, from creating directories and downloading files to launching applications)

One of my projects has a single makefile rule which downloads and unzips a tarball, applies a patch to it, then builds the application within, resulting in half a dozen binaries which are then used in building the rest of the project.

Edit: Ah - I see what you mean now, in your subsequent comment.

> If you want to generate more than one file

A pattern like

    tgt:
     generate_many_files
     touch $@

is pretty common. What's the issue?

> If you want to generate more than one file (like an object file and a module or a precompiled header or ...)

He's not using C, though :-)

> And the problem isn't that they use it, but that they are proposing it as a solution to "everybody".

He's proposing it for the same reason I'm starting to like it, after many years in the industry: as a simple build wrapper.

> And that their Makefile stops working as soon as there is a directory (or file) `build` (or `dev` or ...) in the project root.

And they can fix that problem in 5 minutes, big deal :-)

> Don't forget that not every Make is GNU Make, BSD and other nix like Solaris/Illumos still exist.

This is a very bad reason in this day and age. 99.999999% of *NIX usage these days, probably 99.9999999999999999% for the average person, since most people won't ever get to those environments where BSD and Solaris are still used, is Linux.

And even for BSD and Solaris, guess what... you add an extra step in the build instructions asking them to... install GNU Make.

Heck, even back in 2005 (I think?) for Solaris one of the first things you'd do was to install the GNU userland wherever allowed because the Solaris one was so forlorn I swear I heard wooden planks creak and dust pouring down every time I had to use their version of ps.

And regarding POSIX, meh. If you're a C developer (C++, Rust, I guess), knock yourself out. Most of the stuff devs use are so far removed from POSIX... Actually, not removed, but has so many non-POSIX layers on top (I mean not standardized). Ruby bundler is not standardized like awk. Python pip is not standardized like make. Etc, etc. That's the reality we're in. POSIX is very useful but only as a very low level base most people don't need to chain themselves directly to. I'd definitely not avoid a tool because it's not in the latest POSIX standard (or only in the latest POSIX standard).

> He's not using C, though :-)

As said elsewhere, the use-case in the article is too simple to warrant a Makefile. So: if you aren't compiling some static language, you do not need - and certainly don't want to use - Make.

> you add an extra step in the build instructions asking them to... install GNU Make.

The main reason to use Make is that it is installed everywhere, as stated multiple times in other posts. If you must install something, you can also install a better alternative for your specific use-case to Make.

> one of the first things you'd do was to install the GNU userland

Yes, and the Unix vendors even shipped them on companion CDs or similar.

> is not standardized like awk

Same problem with awk (and sed and ...): some weeks ago I had problem with the SDK for some real-time Linux that works with mawk only, and not with GNU awk (most of the time it's the other way round, only working for some GNU program).

It's a much smaller problem to port a makefile to a different make than to deal with most of the alternatives and their requirements.

That depends if the person who must do the porting knows Make or not and which GNU Make (it's always about GNU Make!) feature had been used. And chances are JS devs don't at all or just as little as the one who wrote the article.

Don't get me wrong: I don't like Make, but I hate CMake and Autotools (and many other C++ build systems) too (and C and C++ and Fortran compilers and their vendors).

> As said elsewhere, the use-case in the article is too simple to warrant a Makefile. So: if you aren't compiling some static language, you do not need - and certainly don't want to use - Make.

I've found that I prefer make as a command runner and most of the time I'm just running Python poetry commands or building Docker containers or running AWS infra commands. It's very useful to have a simple tool to run commands and have them depend on each other.

And regarding many of the alternatives to Make, they're either more complex or have other issues:

https://news.ycombinator.com/item?id=41608555

> This is a very bad reason in this day and age. 99.999999% of *NIX usage these days, probably 99.9999999999999999% for the average person, since most people won't ever get to those environments where BSD and Solaris are still used, is Linux.

You have a lot of confidence. In reality, it's probably more like 30-60%, more now because of WSL. The rest is Mac OS, which uses a BSD userland and hence BSD make by default.

> The rest is Mac OS, which uses a BSD userland and hence BSD make by default.

No. Just a really old version of GNU Make

     make --version
     GNU Make 3.81
     Copyright (C) 2006

Why would they do this? I could understand using a non-GPL make because they hate it, but using an ancient GNU make is just handicapping your users for no gain.

Don't ask me. One could argue that this is the version they included in some early version and have kept for compatibility reasons, but that doesn't make(sic!) sense.

GPL 3.

Apple has restrictions about what software on the system you can modify as a user and how, in the name of security. GPL 3 is unfriendly to such restrictions. Whether what Apple is doing on the Mac specifically violates GPL is, well, a matter of debate that has never been tested in court, but Apple thinks there's at least some risk there, and that the risk isn't worth taking.

This is also why ZSH is now the default shell on the Mac. ZSH never switched to GPL V3, so it was either that, remaining on some god-awful old Bash version, or making their own.

WSL basically runs GNU/Linux distributions so I fail to see the significance of that point.

And for MacOS you do the same thing, you get them to use their beloved homebrew to install GNU Make.

> And they can fix that problem in 5 minutes, big deal :-)

Honestly, a big issue I see is that people can somehow argue with a straight face (and successfully too!) to invest weeks of work introducing a pet project to avoid a 1 hour inconvenience that happens once every blue moon. Proportionality takes a backseat very quickly to motivated reasoning.

Is this post for or against Make? And why is Make not a "pet project to avoid a 1 hour inconvenience that happens once every blue moon"?

It's a general observation on over-engineering, "resume driven design", and proportionality being somewhat of a blind spot in software. But yeah, I'm not going to lie, my brain certainly patterned matched towards "this is going to be a Bazel guy isn't it?". So, Buck2 was close enough. Those are exactly the kind of multi-week pet projects I'm talking about that are too often introduced under vague and disproportional pretenses. Well, multi-month and dedicated specialists going forward are perhaps more accurate for those. But maybe that's the point.

But my argument has been that _Make_ is already too complex for the given task.

And talking about complex C and C++ (to be fair, the complex ones are almost always C++ ;) projects, I would not say that CMake (or Meson or ...) is less complex than Buck 2, it certainly has _way_ more magic than Buck 2. And getting Make & C++ & ccache(or whatever) & distcc (or whatever) to work _reliably_ isn't easy either ;)