Using Make – writing less Makefile
text.causal.agency
text.causal.agency
…but oh no there is a constant stream of inquisitive folk coming over, beer in hand, suggesting cool adjustments / hacks / “improvements” to your setup. With grilling it’s actually pretty easy to defend oneself from this unsolicited advice. There’s only one grill after all. It’s not like it’s some company wide grill that anyone can make changes to solely by convincing one other person to sign off on their PR / MR / changeset.
I’ve seen some very “clever” makefiles (and cmake lists.txt) over the years. It is hacker catnip for all manner of Jackson Pollock ad hoc scripting rubbish and I lament every commit that did anything except make the makefile less complex! All new software engineers should be given a shiny ball to play with when they get bored with their day job, lest they decide to take out their playful creativity on the build system instead.
But it is the opposite of beauty, or elegance, or organization.
Honestly all the millions of man months that have gone into writing makefiles should have gone into making make more usable.
for example:
all the implicit rules in this article? Nobody¹ uses them, they should just be explicit.
You shold explicitly distinguish between a target and a file and a directory. don't let them depend on each other helter-skelter.
indent, tabs, spaces. what a mess.
requiring continuation lines and juggling return codes?
macros that break with whitespace?
I could go on, but ugh.
I actually like cmake better than make. Just having IF statements and a flow makes things more understandable. But yeah, you trade one set of problems for another.
look up "modern cmake" and you'll see the nitty-gritty between nice theory and ugly practice.
[1] for practical purposes
For some reason, people prefer to add more layers on top of Make rather than improving the core language and tool. Maybe there's some good reason?
They want to get in, get it done, and get out.
But who wants to improve the DMV?
GNU Make has been steadily gaining new functions though. I've been reading the manual again recently and there were a few functions I did not recognize: let and intcmp. The new integer comparison function is particularly notable since people have gone to some rather incredible lengths to implement arithmetic in make:
It's also possible to write native plugins for GNU Make to load at runtime. New functions can be implemented this way. It works just like C extensions for other scripting languages. There's also Guile extension support but unfortunately it doesn't seem to allow new function definitions.
Well that’s horrifying. What is the use case? Or is it just hackers gonna hack?
That would seem to be the most obvious and trivial example and I just assume that there must be countless others without having to actually see them and agree that they are valid.
Wait, how could I forget the other most obvious example, comparing version numbers of requirements/dependencies?
The Vala preprocessor can't compare numbers and so I have to implement "if libfoo < x then add -D OLDFOO" outside in the makefile and can only have boolean #if OLDFOO in the code. (libs don't always provide good enough equivalents, maybe only provides a numerical value that you are assumed to be able to compare, or maybe provides nothing at all, or does provide a set of booleans intended to be used for this but they aren't granular enough, etc)
As someone who cares about the aesthetics of code, I thought I would hate using make but I ended up enjoying writing modern makefiles.
When make was created back in the day, I get why certain choices were made, like with virtually all Unix tools of that era. But we don't have to use them in the same way. In general, our tooling is so much better that the so-called warts of make are more like minor annoyances than show stoppers.
* It actually does the complicated job correctly, or can, it provides the means to. A lot of things that claim to be "new better make" are simpler by being simply incomplete and/or internally more magic and less correct and less deterministic.
* Single simple binary. No freaking python or ruby of just a certain vintage and 500 modules but maybe not the system ones or maybe only the system ones...
* Always available everywhere, not just in the middle of the bell curve of modern and popular systems, and will be tomorrow exactly the same as yesterday. Deviations like gmake vs bsdmake are essentially trivial and don't really break anything or present any real problem. IE: portable across both space and time in all directions of all dimensions.
Make is sane for the job it does. It's a great example of a correct use and application of a DSL.
And all those complaints are still true. I address them by:
* Simply adopt a habit or discipline to voluntarily avoid the implicit features. Don't use globbing either. Just don't use those features that are unwise. They are still there, but you don't have to invoke them.
* Just suck it up and deal with the tabs and whitespace and line-continuation and quirks like how each line of shell in a target is a new shell environment unless you build the annoying long single command, capturing exit values so they don't reach the parent make process and get treated as an error, etc. Whatever. To me that's all just normal shell scripting requirements in any context, make not especially different from cron or cgi or other. Rather that than passing around powershell objects or something.
Could be better but ninja or whatever is not the answer to those minor annoyances.
Actually I think it's fine for the implicit automatic features to exist, I just think no one should use them in most cases.
It's fine for a tool to try to be as magic as possible, as an optional thing you can do whaen you decide you want. And for practically any makefile that you are bothering to actually write into a file that you are keeping and running again tomorrow, and giving to someone else to run some other day on some other machine... everything should be explicit. Just like the projects code, it's documentation as much as function, and no part of either the documentation or the function should be left unexplained and undetermined.
But that's a seperate issue from the tool having the ability. I think it's not a tools job to tell me what I want and why and what is and is not valid. If make never had the magic features, I would not miss them feel any need to invest in adding them, but if they are there anyway somehow, I also don't see any reason to remove them or create a new tool just to have something that doesn't have them.
I should be free to write a short super convenient fully magic 3-line makefile that just says "do the likely thing with whatever is in this directory", and at the same time, no project should ever have that as their real makefile. Both stances are valid at the same time, I think.
I don't see `make` as a build tool because I believe from experience that is where the road to hell begins. But it is convenient to have `make` call CMake/Cargo/pip/whatever. Plain `make` can build, `make test` can test, `make fmt` can auto-format... Even better if you alias `m` to `make` in your shell.
For a repo where an easy to install (single binary) dependency is a non-issue, consider using just. [1] You get `just -l` where you can see all the command available, the ability to use different languages, and overall simpler command writing.
Although SCons is Python (which is a pro or con depending upon your perspective), it has strong dependency management. Or is the argument that dependency management is part of build, not general project maintenance?
If you go a bit further down this route, you end up with build tools that generate the compilation rules for you in some form: These are Automake/CMake/Meson and SCons. I did use scons years ago and it was nice, but its definitely completely lost its market share. IIRC Scons does this without generating Makefiles.
Task and Just are following a different route. The problem people have solved by using a "hack" in Makefiles (PHONY targets), so that you can easily run "sub-commands" in Make (make install_deps, etc). It would never occur to me to use Scons in that space.
Btw. a third option is to use a shell script like the following (POSIX-shell compatible actually).
sub_install_deps() {
set -e -x
# ...
}
sc=$1
case $sc in
"" | "-h" | "--help")
sub_help
;;
*)
shift
"sub_${sc}" "$@"
if [ $? = 127 ]; then
echo "Error: '${sc}' is not a known sc." >&2
echo " Run '${prog_name} --help' for a list of known scs." >&2
exit 1
fi
;;
esacYou can invoke it by naming a named rule, not a file, but the logic will remain.
If this is not what you want to be doing, and if your Makefile is full of .PHONY targets, you likely need a Justfile instead, or (worse) plain shell scripts.
I was using Make for running commands, but it interprets all command line parameters as build targets, which was annoying. And Bash scripts in Makefiles aren't type safe, became messy.
Nowadays I use Make only for building. And Deno + Typescript instead, for running commands & scripts.
(Hadn't heard about Just — the scrips aren't type safe though?)
Just runs code written in other languages. It's `sh` by default, but you can define anything (e.g. `python3 -c` to run Python scripts).
Just don't.
And do just :-)
Some people on HN believe that if you use make primarily as a task runner that you're doing it wrong and should use something else. I don't agree.
Most users are using multiple aspects of make: as a task runner and as something that handles the dependency graph when building something digital.
As a web developer, I used npm scripts, grunt, gulp, etc. to manage web builds but now I only use make and have gotten off the merry-go-round of web build tools.
When you create a Makefile, you're describing a dependency graph of what depends on what and the order tasks need to take to produce a particular set of digital artifacts--you're not writing shell scripts.
Turns out there are all kinds of edge cases and foot guns that make handles for you that you'd otherwise have to deal with if you wrote a bespoke shell script.
Plus make is battle tested and has been around since the dawn of Unix--there's literally no build scenario it can't handle.
I hadn't used make until a few years ago but it's been one of the best investments I've made in my workflow.
Well except this little case where some files or folders in your paths have spaces in them...
For example:
A) You have a recipe that installs dependencies if they're not up-to-date (such as "pip install").
B) You have a recipe that compiles stuff if it's not up-to-date; it's made to depend on A.
C) "make" can be an interface recipe that depends on B and does nothing else.
D) "make test" can be an interface recipe that depends on B, then runs tests.
E) "make run" can be an interface recipe that depends on B, then actually runs the code / a webserver to interact with.
Nowadays whenever I put one of these together there's 2 versions of A (one for python, one for node), 1+ versions of B (webpack build without watch, maybe others depending on the project), 3 versions of D ("test-js", "test-python", and "test" that does both, and each of them only requires the relevant parts of A and B).
Makes it trivial to ensure you're up-to-date after a "git pull" without having to waste time waiting on things that don't need updating.
Like you, I use Make as a front end to lower level build tools. It seems to fit that role well.
I get it, I do. I know there's all sorts of legacy reasons, but my 4k resolution screen with 12GB vram and 32 cores still runs make build serially, outpitx to an 80x24 window and is bottlenecked down by stdout flushing, all because we can't change defaults.
I think cmake got this absolutely right, fwiw. You define the version of cmake you're writing against, and they go to great pains to ensure that they preserve behaviour even in newer versions of cmake. As an example, we could have a special variable:
make_ver = 4.5
This would let make do things like accept spaces instead of tabs, be parallel by default, etc in future versions while presenting old behaviour.
If your build is extremely simple and fast then it cannot add value.
Sure, make syntax sucks. It's arcane and hard to google much of what is going on. (Shower thought... a tool that lets you view a Makefile, giving explanatory tooltips for syntax elements would be great).
But make is widespread, ancient and eternal. It isn't going anywhere, despite decades of new tools trying to take its place. It does many jobs relatively well.
I'd go so far as to say that the platonic ideal of development is:
make # Build the project
make install # Install the project
make clean # Clean the project build dir(s)
and whatever new tool you use to actually build things should live behind make. You should have a really good reason to deviate from this. It's approachable and familiar to most devs(well, ok, older C/C++ devs). Sure, write your own custom tool that builds everything in a chroot jail or docker container, has sub tools for installing, running tests, etc... but please, put it behind a make frontend.
> make # Build the project
> make install # Install the project
> make clean # Clean the project build dir(s)
IMO, that last step of cleaning is very not platonic ideal. However... maybe a reality of incremental development?
The first step for me in a lot of cases is just trying a project to see if it's worth my time, typically that would just have been:
cd /tmp git clone http://foo/some/project.git cd project # build steps, perhaps in a good case your make example above, a dockerfile, or nix flake
These days though I'm ecstatic when I see a Nix flake because I can reduce that to:
nix run github:some/project
nix develop github:some/project
cd /tmp
unpackPhase
genericBuild
I guess all of the above is to try and express my different idea of platonic ideal here and how Nix gives it to me (and could perhaps to you, for a cost).
Agreed--make is eternal. One thing that'll probably be true: in a post-apocalyptic future, make will still be around and still be useful.
Yeah. The GNU Make is literally everywhere, it's too useful and it's got too many good features. GNU Make is worth learning and using just because of this. Having to deal with non-make systems is seriously annoying.
> It does many jobs relatively well.
I wonder if anyone else other than me was insane enough to try and use GNU Make to manage dotfiles? I even blogged about it.
https://www.matheusmoreira.com/articles/managing-dotfiles-wi...
Gotta be really careful with stuff like that. GNU Make is simultaneously a rather lisplike metaprogrammable language and a turing tarpit. I screwed up a personal project once because I got sidetracked essentially reinventing a fraction of autoconf in pure GNU Make code.
Now I try to keep things as simple as possible. I still understand the makefile so I suppose it's simple enough.
Not a bad idea IMHO… thanks for the blog post. I'm tempted to try something similar.
https://github.com/matheusmoreira/.files/blob/master/GNUmake...
The metaprogramming template I described is used to implement XDG Base Directories. The links take the XDG variables into account while the real files live in their default locations inside the repository.
I think an important thing to know about make(1) is that you don't need to write a Makefile to use it. There are default rules for C, C++ and probably Fortran.
OBJS = foo.o bar.o baz.o
foo: $(OBJS)
$(CC) $(LDFLAGS) $(OBJS) $(LDLIBS) -o $@
foo: foo.o bar.o baz.o
This would only make sense if the only rule that needed the list was that one that builds, no install/uninstall/package/modify/audit/hash/clean/etc, which is never the case. You also don't want to use globbing in place of an explicit list either.
FOOOBJS = foo.o bar.o baz.o
foo: $(FOOOBJS)> The example Makefile in its entirety:
CFLAGS += -Wall -Wextra
LDLIBS = -lcurses
OBJS = foo.o bar.o baz.o
foo: $(OBJS)
$(CC) $(LDFLAGS) $(OBJS) $(LDLIBS) -o $@
foo.o bar.o: foo.h
clean:
rm -f $(OBJS) foo
CFLAGS += -Wall -Wextra
LDLIBS = -lcurses
OBJS = foo.o bar.o baz.o
foo: $(OBJS)
foo.o bar.o: foo.h
clean:
rm -f $(OBJS) fooPersonally, I'd have gone with
foo: $(OBJS)
$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS)
But that is somewhat idiosyncratic, and based on some half-remembered ideas of second-hand stories of how compilers used to process their command parameters back in the '90s (and before). Those ideas probably aren't relevant now (and may not have been entirely correct originally), so if the recipe in the original article works for the author, I'm not going to say they're "wrong" for doing it the way they did.
You can include in the project Makefile a generic Makefile that may contain a definition like:
OBJS := $(CPP_FILES:.cpp=.o) $(CXX_FILES:.cxx=.o) $(CC_FILES:.cc=.o) \
$(C_FILES:.c=.o) $(SS_FILES:.S=.o) $(S_FILES:.s=.o) $(F_FILES:.f=.o) \
$(L_FILES:.l=.o) $(Y_FILES:.y=.o) $(RC_FILES:.rc=.o) $(O_FILES)
The automatically defined OBJS can be used for linking, and a similar list without $(O_FILES), i.e. without .o files that are source files, not intermediate files, can be used for cleaning.
The makefile quickly gets complicated if one wants an organized source tree such as:
source/module/file1.c
source/module/file2.c
source/main.c
source/tools/tool.c
build/$arch/prerequisites/module/file1.d
build/$arch/prerequisites/module/file2.d
build/$arch/prerequisites/main.d
build/$arch/prerequisites/tools/tool.d
build/$arch/objects/module/file1.o
build/$arch/objects/module/file2.o
build/$arch/objects/main.o
build/$arch/objects/tools/tool.o
build/$arch/executables/main
build/$arch/executables/tool
https://make.mad-scientist.net/papers/multi-architecture-bui...
.SECONDEXPANSION
foo: $$(patsubst %.c,%.o,$$(wildcard *.c))
$(CC) $(LDFLAGS) $^ $(LDLIBS) -o $@
I make small Makefiles like this that ensure my local build environment is correct for the package, then call cmake.
I know we don't do general-purpose "I like this" comments on HN, but, as a Christmas treat to myself, I'm just going to say how happy this little sentence fragment makes me.
As it happens I used to program in both Prolog and TECO, in the same period of time but of course not on the same code bases.
.SECONDEXPANSION
foo: $$(patsubst %.c,%.o,$$(wildcard *.c))
BSD makefiles do look better.
In decades of programming in very varied environments I have never compiled any software project otherwise than by using globbing, so that I have never needed to waste time to write the name of any source file in a Makefile.
I have never seen any reason to do otherwise.
For instance, using special compilation flags for a certain source file, different from the others, is something that I consider a mistake. Whichever is the goal, it certainly can be achieved by other means (e.g. pragmas or attributes).
Moreover, the dependencies for any file must always be generated automatically, they must never be written explicitly, which removes the main reason why in bad Makefiles the names of the source files are written individually.
You must also be aware that they are compiled in a different way and you must search a frequently too big and obfuscated Makefile to discover which is the applicable compilation rule.
Unfortunately I have seen many projects that had used such tricks and it was always painful for maintenance to discover how they were supposed to work.
Once you start getting to multiple directories and recursive scans, the build time begins to grow enormously and usually unexpectedly.
Not a problem at all for small or even medium size projects on a fast machine.
But once you start building very large projects with multiple sub-projects, it generally slows the machine to a crawl on every build.
The bigger a project is, the more important is to use only globbing and never write explicitly any file name lists.
However, I believe that this, the slowness of globbing for big directories, might be a strictly Windows-specific problem, if it exists. At least on Linux and with XFS (which has B-tree directories) I have never seen globbing to take any noticeable time, even on directories with ten thousand files or more, where it still appears to be instantaneous at the human reaction time scale.
There is no reason to ever do recursive scans. All the directories with source files of a project must be scanned to search for any source files located there, but only once.
Globbing is a security and reliability hole. You don't ever really know what got included or will get included, you just assume you do, and you are right only most of the time and only by luck not by actually ensuring it.
Saying things like "only the files I expect should ever be in this dir" is just wrong swiss cheese thinking. (you never said that, but another commenter essentially did)
Alternatively, you could move such a file to a subdirectory "Attic" or the like.
Any of these solutions would ensure that globbing will no longer include that file in the list of source files, without needing to update any project configuration files.
It appears that almost nobody reads the manual of GNU make, despite the fact that it is extremely instructive.
I have read the GNU make manual once, about 25 years ago. Then I have written a set of small Makefiles that I have used in all my software projects forever, until now, with only extremely small changes during the years, e.g. when new compiler options have appeared, or when I have added compilers for additional CPU targets or operating system targets or additional programming languages.
It is possible to make such generic Makefiles that will work in any software project.
For example, in the simplest case, when the source files are in a single directory and you use that directory also for building the project, copying a small template Makefile that includes the appropriate generic Makefile for the target CPU and operating system is enough so that "make" will identify all the kinds of source files that exist in the directory, generate the necessary dependence rules, invoke the appropriate compilers and build the executable.
For any more complex project, only a minimum of information needs to be added to the template Makefile. For instance, when the build is done in another directory than the one with sources, or when there are multiple directories with source files, a list of directories with source files must be added into the template Makefile. Lists of non-standard libraries, non-standard library directories and non-standard include directories may be added. Additional options can be specified, e.g. building a shared library, not an executable file.
Besides these, nothing specific to the project needs to be written. Adding or deleting source files or renaming source files do not need any changes in the Makefile, but after such changes a "make clean" is recommended.
Instead of this simple and obvious approach, more than 99% of the software projects that I have seen use absolutely horrible huge and unmaintainable Makefiles, or they use "make" replacements like "cmake", which are much worse than the traditional "make", so I fail to understand why anyone would want to use them.
How does one discover dependencies in a cross platform way with Make without writing the logic themselves to stay up to date with platform changes? Or picking up configuration options from dependencies.
How does one make use of Ninja with make? Or discover changes to sdk paths for new platforms?
You end up having to duplicate that logic across every repo that needs it. Along the way you accrue lots of variance.
In the end you get something that is difficult to debug and scale. That’s why so many projects moved to cmake. It scales better. In much the same way that you can technically do anything you need with C, but other languages add ergonomics and scalability that people value more.
I have stopped using MSVC many years ago, so I do not know how it handles dependencies, but with gcc or clang that works regardless of platform, with compiler-specific options.
Moreover, software for one platform can be built on another platform. What matters is only the platform used for building, where the compilers are hosted, not the target platform for the built software.
In general, I extract all platform specific definitions, like the names of the compilers and their command-line options in platform-specific Makefiles, which include a generic Makefile with platform-independent definitions, rules and make targets. Any platform-specific details, like SDK paths, are encapsulated in the definitions of certain "make" variables.
While I have not needed to update the platform-independent generic Makefile for decades, the platform-dependent sections need updates from time to time, but that is not done for every software project, but perhaps once per year or more seldom, when significant new compiler versions, standard libraries or other new software tools become available.
I have not seen yet any reason for using Ninja. Perhaps it may be faster when building something like chrome, but because chrome is likely to be the slowest-compiling software project known to mankind, it is hard to tell how much is gained by Ninja. For software projects of more typical sizes I have not seen noticeable advantages of Ninja. Traditional make can keep all the cores of a CPU 100% busy, so there is no way any other building system can be appreciably faster. There are some very bad Makefiles that are intrinsically slow, but that is a problem of those Makefiles, not of "make" in general.
Using a minimum project-specific Makefile section can only simplify debugging, not making it difficult.
I have never seen any evidence for the claim that cmake scales better. On the contrary, at least in all the open-source software projects that I have ever seen cmake is a major source of bugs in the building process that nobody knows how to fix. (i.e. after some software updates cmake fails to find files that exist, presumably due to some errors in the cmakelists that appear to be arcane enough that they are hard to find) I have never seen in projects that use traditional make such errors that are so frequent in all the projects that use cmake.
I have a lot of experience in building software projects, because for many decades, with the exception of a few professional programs without alternatives, I have been using only programs that I compile from sources. Most projects have very bad Makefiles, which are very hard to maintain, i.e. to change when files are added, deleted, moved or renamed, but at least they build the software projects reliably. Whenever cmake is used, building a new version is always an adventure with unpredictable outcome.
And make tends to fall significantly behind Ninja in my experience for cold builds. Here’s a post from someone else that echoes my experience https://david.rothlis.net/ninja-benchmark/ but for a sufficiently complex project even a well tuned make build is about 20% slower on CI, which is about 10-20m per build.
It also seems like you’re giving make the benefit of the doubt when it comes to “most being badly written but my own is great”, while simultaneously deriding cmake without giving it the same benefit.
But to your larger point, and going back to my C vs other languages comparison (let’s pick C++ for arguments sake). Yes Make can do it all, but even by your own statements it requires doing everything correctly and there’s significant room for error. CMake makes the trade off of abstraction for a more consistent experience with minimal work.
This is similar to C and C++, where yes C can (1) do much better in the very very specific right hands, but (2) it can also become an unwieldy mess. C++ with RAII may seem perplexing from the lens of only accepting the former and not the latter. It lets people spend their time elsewhere on the system.
So reading all the makefiles should take a negligible time, even in a big project.
I have seen too many projects with huge makefiles and with much more makefiles than necessary, but all of those are examples of misuse of make, which are easier solved by an efficient use of make, instead of by replacing make with another tool.
For instance, there are many projects with a Makefile in each directory with source files, which is a very bad choice that multiplies the number of makefiles. Makefiles should exist only in a dedicated build directory, not in source directories, except for very small projects with a handful of source files.
The best way IMO, is to have a single very small Makefile for each final file that must be built, e.g. executable file or library, in a dedicated subdirectory of the build directory. This allows for a maximum simplification of the Makefiles and their number and size is independent on the number of source directories and on the number of source files.
It's also doing a lot of variable expansion and if the makefile uses macros like define....endef to generate rules it has to expand those.
Make is also often trying all sorts of pattern rules to see if they will allow it to fulfill dependencies. This takes time.
If makefiles are small and you run make on each one separately then you miss dependencies across modules and you get the syndrome where program A doesn't get rebuild when library B changes.
So there's a price to pay for everything. Ninja is a lot faster because it just doesn't have some of these features and you can make a big makefile where dependencies work properly with it and not pay the price of slow parsing.
I worked on builds that took 12 hours on a large cluster of build machines where parsing was 45 minutes - so it matters. The more general solution would be to use a packaging system like the Linux distributions but even that doesn't save you from every problem as I also found out when working with OBS on a linux phone distribution.
However, the same link shows that even for relatively large projects the time difference is less than a second, which matches my experience.
Ninja requires much more effort for using, as it is not a complete build system, but just the execution component, and except for projects of chrome size that effort is not worthwhile for a subsecond gain in certain scenarios.
I cannot imagine how a "tuned" make build can be 20% slower, except when the project is not really tuned, but it is badly organized. A "tuned" Makefile means that the execution time for "make" is negligible in comparison with the execution times of the compilers and linkers. The time spent in "make" should be measured in seconds an most, never in minutes.
Only a very slow file system can cause "make" to be much slower than Ninja, because the dependencies for "make" are stored in many ".d" files, equal in number with the source files. On modern SSDs or RAM disks, that is never a problem.
Perhaps you are right about CMake. I cannot be certain whether CMake is good or bad, because I have never created a CMake project myself, I have only built CMake projects created by others. Nevertheless, there certainly is a problem with the CMake users that I have not seen yet one that can explain which are the advantages of CMake. All the tutorials that I have ever seen about CMake were showing how to do in a complex way things that can be done in a simple way with GNU make, so I never had any reason to investigate any further.
What I know for sure is that much fewer understand how to use CMake than how to use make, because the frequency of bugs in CMake projects is much greater.
I would like to see how CMake can provide minimal work. With GNU make, creating a new very simple project needs only copying a template Makefile in the source directory. For a more typical software project, the template Makefile must be edited to add a list of directories, those that must be searched for source files (I actually write the list as a prefix directory plus a list of subdirectories for it, as that is how I normally structure the projects).
For external dependencies, up to 3 lists may be added, of libraries, include directories and perhaps of library directories, which should suffice.
That is all. Nothing more. I normally build each target file of a project, e.g. executable or library, in a separate subdirectory of the build directory, so by default it is not necessary to write it in the Makefile, because the name of the subdirectory will be used for the target. The root of the build directory contains a Makefile that I never change and which descends in each subdirectory to build its target.
How can be CMake more simple to use than this? All CMake projects that I have ever seen were much more complicated and they were very difficult to modify, with a very large number of project-specific details that have no place in the project configuration files, because the building system should be able to deduce them.
All the examples of using CMake that I have ever seen have demonstrated a tool with a much lower level of abstraction than GNU make and which is tedious to use.
GNU make does not need any such thing like the CMakeLists that are ubiquitous with CMake.
Then what is a Makefile?
And if your dependencies only amount to looking for source files, then that’s fine. Many are more complex than that and include configuration options themselves.
Again, I think you’re hyper indexing on just your own setup and ignoring anything outside of that, while simultaneously hand waving the number of bad makefiles out there and any shortcomings to drive home that Make is simplest.
Meanwhile by your posts, you’ve had to create an entire scaffolding of support around your makefiles to make it appear that simple. How is that any different than CMake or any other build system/generator? In the end you’ve just orchestrated your own version of such a thing and are decrying any other system as overly complex or just needing adjustments.
I believe we’re just talking past each other, but I’d really encourage you to consider that the reason you’re confused why people don’t just use Make is perhaps well founded and rooted in the number of things you have outright dismissed or don’t care about for your own projects. I’m not even specifically advocating for Cmake, but there’s no great mystery why so many projects use it or other build systems; it mostly lets them shift the responsibility of configuration to the system and focus on their task at hand. Again, exactly the same as C vs C++/Rust/etc…
CMake script syntax is not very great, and debugging is no better than sprinkling printf's throughout... So, why not use Autotools? /bin/sh has been the norm for most build processes, to the point where the later-designed YaML syntax is _effectively_ the SAME as a Makefile with /bin/sh statements running the pipeline procedures.
CMake came along and tried to fix complexity in Autotools while adding kitchen-sink baggage along the way.
I was merely pushing back on the person perplexing on why people don’t just use Make.
meson lags behind for supporting new features like different languages (Swift, ObjC etc), Xcode/msvc updates, IDE integration (CLion is amazing), build outputs like frameworks.
basically, Mesons better ergonomics lose out to practicality for my use cases. To get it do what I need, I’d be making my own meta system around it.
While there is certainly value in using CMake (as so many projects have chosen it as their build system)—it also becomes a sort of cautionary tale whence projects try to reinvent the wheel to fix some deficiency, only to build a complex system with it's own set of deficiencies. Obviously, part of this is due to the fact that commercial entities (cough Microsoft) deviated from any sort of standards-based approach to building software.
But for every build-configuration system, Autotools seems to be the only one which dictates that a developer machine merely have POSIX tools installed. Nowadays, that's all the big players, considering that MSYS2 on Windows is better than ever. And it supports many languages to boot: C/C++, Objective C/C++, Go, Fortran, Erlang.
It all depends on what you want out of the system - the ability to build against multiple versions of a specific linux distribution or compatibility across unix or even to non-posix operating systems. So the complexity exits from the generic parts where you say:
PROGRAM:= bob
SOURCE:=fred.c
LIBRARIES=gtk
include build_program_template_$(PLATFORM).mk
If you're the one building the code and deciding where to do it that's cool. For people who want to port your code to some other platform, cmake or autoconf will help discover what needs to be done on some specific plaform, check that the needed features/libraries are on the platform and construct a build with all the features it can support on that platform turned on and the ones it cannot cope with turned off.
They automatically do part of the job you might be doing manually when maintaining templates. This is useful when you're giving the build to someone who just wants it to work and doesn't know the ins and outs like you do. But of course they are complicated and for all the great functionality they are far more complicated to fix when they don't work :-D
When compiling a project once from a clean state make has no advantage over a shell script that would use a tool like "parallel" to distribute compilation jobs over all available CPU cores.
However writing such a script for parallel compilation is more complex than writing a simple Makefile executed with "make -j N".
As you say, the main reason of existence of "make" is the acceleration of the recompilation of a project after incremental changes in the source files, when only the files that depend on the changes are recompiled, saving time.
Not to take away from your correct answer of course
Now imagine that you want to run individual steps from time-to-time, instead of running your whole script start to finish. So maybe you write functions that represent each group of commands, and call the function based on an input argument.
Now imagine that some of your build-steps have dependencies on other build-steps, and you want to be able to say "run step X, plus whatever else you have to do first." Or maybe you want to retry a failed build from wherever it left off, instead of having to re-run the entire thing.
Now imagine that you'd like to have some level of isolation between build-steps, so that one step doesn't mess the other one up.
As you add more and more things to your build script, you'll find yourself writing an implementation of Make pretty quickly.
Now
- sometimes - your build fails halfway though, and you want to resume from where-ever it failed. S
The more possible variations you have to cater for the worse build scripts get.
If you try not to redo work that you did 5 minutes ago that's still valid then build scripts become even more complicated whereas the makefile doesn't change at all.
Other tools do declarative even more than make does but as they get easier to understand they tend to lose abilities.
It was fine in its era, but that's a long time ago. It is completely unsuitable as a project management tool and a poor task runner.
Also its ubiquity is overhyped. There's no build platform where I can't download the tools I need, that's the entire point of a build platform, and most such platforms come with far more advanced tools pre-installed.
So it gets used a lot and it's only a language for dependencies and rules. It doesn't do "project management". Its the simplicity of what it's trying to that saves it from ever becoming irrelevant - because it can be made to fit almost any use case. It's not a special tool for enforcing one structure or building only one language - something that seems highly regressive to me but which is adopted by many languages now.
This is a non-priority, an unreal use case. I can't build GCC without a functioning C++ compiler and a fairly sophisticated OS environment and that's fine. Real-world use cases for bootstrapping a build environment from rubbing two sticks together are so exceptionally rare as to be near fictional.
Even if we allow that such cases exist, they are unicorns, not a thing to design tools around.
> It's not a special tool for enforcing one structure or building only one language - something that seems highly regressive to me but which is adopted by many languages now.
This is what makes it so unsuitable. A tool that can make no assumptions about its application is less and less useful a tool. A bread knife is better at cutting bread than a plain 10" kitchen knife, a boning knife better for deboning, etc.
We live in a world where for every language there are build tools that know far, far more about the needs of that language environment than make, and thus are far better suited.
The "assumption-making" build tools are the unicorns that inevitably cannot dominate because they're not generally applicable. The more they assume the more niche they become.
So you might come up with some new thing that "Works For Me" and now all those people who are peacefully using the tool on their whatever platform have to fiddle with it to make it work again - or the person who made it work is no longer around and that platform loses support for the new versions.
When you say "cultural" you make it sound like people do things because they are ignorant and resistant to change but it's worth at least considering that their point of view is quite different.
They already do dominate. In the last bastion where make can be said to be popular (C) make has already lost to CMake and its usage shrinks every year [1]
The situation is moving even faster for C++ [2]
And for literally every other language in the systems programming space (C#, Swift, Rust, D, Go, Zig), make was never a player to begin with.
[1]: https://www.jetbrains.com/lp/devecosystem-2022/c/#which-proj...
[2]: https://www.jetbrains.com/lp/devecosystem-2022/cpp/#Which-pr...
There's nothing fun whatsoever about fixing build problems with cmake - because you have to understand make/ninja AND cmake. It's slightly easier to understand than autotools, though, where one has the same problem.
That make happens to be a possible target of CMake is irrelevant to this discussion, it replaces the usage of make. make becomes an implementation detail.
No one should be writing Makefiles. You also shouldn't be using make as the underlying task runner but for a different set of reasons.
> There's nothing fun whatsoever about fixing build problems with cmake - because you have to understand make/ninja AND cmake
You need exactly zero understanding of make or ninja to use CMake. Ninja explicitly is not meant to be understood by end users, and only as a target for generators like CMake:
>Where other build systems are high-level languages Ninja aims to be an assembler ... it is designed to have its input files generated by a higher-level build system [1]
You cannot name a single use case where it is beneficial to know the mechanics of make when using CMake, much less the mechanics of Ninja.
You would use compile_commands.json for this, which would show you the literal commands being invoked. What program is doing the invoking is 100% irrelevant. If the invoking program is what is breaking your build, your build is screwed beyond comprehension.
You don't need to know LLVM IR or the GCC intermediate representations, you don't need to know the internals of the ELF format, and you don't need to know make.
If you want to know these things more power to you, but don't insist that everyone should be writing LLVM IR because it's a more general building block than C++ or that anyone should use make for the same reason.
Languages don't exist in a vacuum.
A big reason for the resurgence of make by developers is every language/environment comes with something new that tries to recreate make. They all have issues and limitations where make ends up being a better tool.
Also, when you need to deal with multiple languages in a project, these tools tend not to be that helpful.
As a web developer, there's been so many attempts to make tools like gulp [1], grunt [2], npm scripts [3], web pack [4], etc. and a dozen more, to do a part of what make has been doing for decades. And literally every 6–12 months, there's a new, hot tool that everyone gets excited about. It's just reinventing the wheel over and over.
Now that I've settled on make for web projects, it's no longer a concern.
I much rather describe my dependencies using make than JavaScript (ugh), which many of these tools use. Some tools fall out of favor; developers stop creating plugins or whatever for them. As a user of the tool, you can find yourself shit out of luck for your particular project or use case.
The beauty of make is it can handle the tasks of building a web site or web app using modern tools that didn't exist when it was created. As someone mentioned further up, make is eternal; it's not going anywhere.
[1]: https://gulpjs.com
[2]: https://gruntjs.com/
I have good news for you: this is already the case! In all modern versions of make you can use semicolons instead of tabs.
You just write
target : dependencies ; rule
target : dependencies
^Irule
Many times, a Makefile snippet in an HTML page or a PDF file has had the tab character auto-magically morphed into spaces. So when it is copied, it will be a syntax error in a Makefile. Even copying Makefile snippets from one terminal window to another will eat those tab characters. Fortunately my vim editor is configured to handle most of those edge cases now, but it took me... I don't know... 10-15 years to get that right.
My personal rule is that control characters which are visually indistinguishable from other characters have no place in source code which are meant to be written, read, and copied by human beings. So yeah, that includes the <tab> character. And yes, I do think the Golang people got that wrong, although the 'go fmt' command largely solves that problem. Except when reading Go source code on web sites which don't handle tabs well, and you are nested 6 levels deep, and half the code is unreadable because it's bled off the right side of the page.
This is a text editor configuration issue, not a character choice issue.
As you say, a small elegant makefile is lovey.
That is not the sort of makefile you get from the autoconf/automake family, and (ha!), good luck if something goes wrong.
The reality is that a small simple makefile is not sufficient to build real software.
The problem is fundamentally that make doesn’t compose well.
Most real build systems such as cmake, scons, cargo, etc. provide primitives for dealing with complex messy situations like “oh today I’m using visual studio not clang” or “does this platform have stdbool.h?” or “is the flag for disabling a warning different on this compiler?”
…and a way of composing that tasks into smaller ones (eg. add_subdirectory() and find_package()).
Make does not.
Make is a simple tool for simple tasks.
The reason people don’t like it, in my experience, is they have had to work with a make based project that has grown beyond the trivial size and it’s become a nightmare.
It makes easy things easier, and hard things much harder.
If worked on SPAs where they used make instead of webpack; but it’s a stupid solution. It doesn’t do live reloading, or any of the other pipeline stuff… but by gosh they tried!
You know what I like about clojure? It’s a simple tool that scales well for both simple and complex tasks.
Make simply doesnt.
…and that is reflected in the reality, which is that increasingly build tools are moving away from it and towards others, even for the low level tasks that people used to “generate makefiles” for eg. to ninja
I really am not a cmake fan; but there absolutely no denying it works very well in many real world situations, where trivial “-lsqlite” flags in a naive makefile don’t and cant work.
plan9 exists in an enviable position of only needing to build in a few controlled environments.
For example, http://9p.io/sources/plan9/sys/src/libmach/mkfile reads:
CFLAGS=$CFLAGS -I/sys/src/cmd
/shrug
Try reading the llama.cpp makefile, which is an example of an excellent cross platform makefile that works very well: https://github.com/ggerganov/llama.cpp/blob/master/Makefile
^ this is what many real Makefiles look like, and this one is excellent and well maintained.
Many are worse.
Make is good when you only have easy problems to solve (like, specifically, only having to build one specific constrained environment for one specific compiler).
You can use make in this form without a makefile even:
~ % echo '#include <stdio.h>\nvoid main() { printf("OHAI\\n"); }' > ohai.c
~ % make CFLAGS=-DDEFINE_ME_A_MAKEFILE ohai
cc -DDEFINE_ME_A_MAKEFILE ohai.c -o ohai
~ % ./ohai
OHAII also use its metaprogramming and late evaluation capabilities a lot, heavily inspired by the tricks Buildroot is doing.
If you are mindful of its restrictions (it works with “words” separated by spaces, thus paths with spaces in them are a problem unfixable without severely breaking backward compatibility; and your targets need to be expressible as files with meaningful modification timestamps), it’s a very powerful tool. It deserves more appreciation.
Not every manpage was great, but many were, and many could also be formatted, printed out, and placed into a binder for easier reading.
What I enjoyed most about manpages is that the people that cared to write good documentation on things like bash.
If you have groff installed you can:
man -Tpdf man >man.pdf
Specifically, in one of the examples, to list all dependencies as the input to `cc`, you can write `$^` instead of `$(OBJS)`.
I'm not a fan of magic symbols. Hard to remember, especially across languages.
$@ -> the target (like an email address)
$^ -> the dependencies (look up)
$< -> only the first dependency (look up sign, but rotated 90s left to point at the first one)
Nowadays I use CMake and am really happy with it.
These days I just use Rust instead.
Might I suggest “got-restore-mtime” for this, upstream in Debian and Ubuntu already.
Did you really spend hours on this?
Also I think it took quite a while to even realize the mtimes were the issue from an error messages, as I was not very knowledgeable about autoconf / automake.
I love make and still spin up a minimalist Makefile like in the article regularly for tiny projects, but I'd hate to have to actually maintain one for a real world project.
.depends: $(SRCS)
$(CC) $(CFLAGS) -MM $(SRCS) -o .depends
-include .dependsThe problem is that now the tools aren't designed as composable bits that you can really parallelise with a makefile. C/C++ are designed that way and if they didn't use header files I'd consider them perfect from the build system's point of view :-). Golang and java for example are a nuisance.
They prioritise various features over productivity and I think that's a mistake.
Only recently have I come to be kinda fond of its insanely simple syntax but really make hasn’t been a requirement to native development since as long as I’ve been around getting paid.
Yeah, make is kind of neat because it has this functional what_I_want : how_to_get_it syntax, but it doesn't actually work that way. It's a huge waste of time to try to get it working so unless you meet the following caveats, make is not for you:
- you love trivia like "who was the 19th president of the US?"
- on a road trip, you stop to look at historical markers
- you love steampunk/tube amplifiers/making pasta from scratch/listening on vinyl/home brewing beer
If those sound like you, you're the type who won't regret wasting a couple days getting make running and getting cut on the sharp edges. Most of us who just want to get there will be better with cmake, meson, or hand-rolling a script using a modern language (inb4 "make does more than compile code" - yeah, so does ruby or python or any other script, without making false promises or wasting your time)
Can you actually articulate what those sharp edges are?
As for trying to use a "modern" language to write scripts.......well if they're not declarative and don't understand dependencies then how can they be better?
make is no paragon - it just contains solutions to lots of problems that exist in build systems. The fact that those problems exist and make has ways to solve them isn't make's fault. Everyone might perhaps want builds to "just work" so they can ignore them and such people write scripts and use the concept of "rebuild and clean" a lot and it works until the build gets big and slow.
"modern" tools get written with many simplifying assumptions and become slowly complicated as they hit all the problems make hit long ago and try to find ways to solve them.
They are both a wash: you're spinning your gears trying to get a simple task done. My claim is that you might as well do it in python instead of wade through the cesspool of hidden rules or try to debug why your makefile silently fails. It's the same amount of time spent, you might as well check the date stamp in python and rebuild the source if it's newer than the output. They both suck, might as well suck in a modern language than trial by fire with make.
I currently work with people who think a huge build.sh is the way to go - so the build does a lot of unnecessary things just to be sure they are uptodate and it's slow. It's still not bad enough to kick up a fuss but it's going that way.
(do we have anyone who reads HN over a morse clacker?)
What do you have against people different than youself?
(mobile I agree would be hopeless with zoom [I avoid reading on mine even without zoom], but does your desktop zoom mean even TFA's vt100-friendly 72-chars needed reflowing?)
I guess it's fitting for a tool (make) that is also unergonomic by today's standards.
This site simply isn’t as legible as it could be. I am not suggesting it needs to be “pretty”. A monospaced Markdown file would be more legible.
Then a beginner could see what happens in a straightforward way, and then import the C rules. And we could all stop importing the VCS rules which aren't useful anymore.
I have a whole folder of personal tools I’ve written in bash, and had a similar set for make. Some of them I don’t want to touch, but when I have to, I just rewrite them in Node with a little io/ipc helper library I created and it feels like a breath of fresh air.
I tried typing them in my command line (macos 12.4) and I would need to install them to use them.
I was taught make in CS101 in college and the majority of other people's projects I've looked at use it.
So the advantage is ubiquity and familiarity. Which can be useful if your code base is going to have a large number of people working on it.
Then you also stopped learning about developments in your field 15 years ago.
> I tried typing them in my command line (macos 12.4) and I would need to install them to use them.
Ok? And? We don't pre-installed dev tools on consumer operating systems. You won't find valgrind or vcpkg either.
> I was taught make in CS101 in college
The tools we taught to beginners over a decade ago are perhaps not suitable in all cases, in fact they are suitable in very few cases. Not much Pascal usage either anymore.
> the majority of other people's projects I've looked at use it.
Then you live in a very tiny bubble.
There's no way to respond other than to point out you've had a very bizarre, isolated 15 years if the only thing you encounter regularly is make. In the one language ecosystem where make is still popular (plain C), it's a minority that only comes in second overall for build tools [1]
[1]: https://www.jetbrains.com/lp/devecosystem-2022/c/#which-proj...
There’s value in not overloading your cognitive capacity with the most recent fads, there’s enough serious tools and tech to learn.
In so much as neither is pre-installed on a consumer operating system, yes, they are obviously equivalent.
> There’s value in not overloading your cognitive capacity with the most recent fads
Ninja has been the standard backend for CMake and other meta-generators for 11 years, it is hardly recent or a fad.
Moreover, you don't need to know anything about it, since it's a task-running backend format. You need to "know" it as much as you know the ELF object format or LLVM IR, your tools use it not you personally.
Do you need those features? You'd have to read about make to find out but it boils down to there being not much performance difference for small builds so make might have an edge there. When you get to android sized builds the reading of makefiles can take minutes and ninja wins there enough to make it worth sacrificing the features.
Though you could also argue that writing makefiles by hand is equally foolish. Use a tool designed for the job and not some duct tape that “works” everywhere but barely keeps things together. The example in the OP is missing handling of headers #including other headers, just to give one obvious example.
how many people are using systems like cmake (or worse) who would be fine with just make (used right)?