Hacker News Reader

Comment by saagarjha | Hacker News Reader

Considering that all build systems must keep a dependency graph internally, it’s surprising how many of them make it incredibly difficult to inspect why things are being rebuilt. I don’t really get it either, I’ve talked to a lot of people and getting feedback on how the build system is doing its task is one of the biggest complaints I see (others are “what is the build system doing” and “why does this have the worst syntax and weird edge cases”) and I mean all the build tools have literally everything they need to show this externally, but basically none expose it in a reasonable manner (if at all). I should be able to go, “oh you rebuilt all of libFoo.a because I accidentally made it depend on autogenerated.cpp”. If you are making a new build system today (and I am sure some of you here are considering it :P) please make this one of your priorities when designing the tool.

View on HN

GNU Make is pretty good about telling you why things are happening with -d

If you can get away with it use -rR to skip builtins, it makes the output of -d much nicer.

I usually end up running make with -rRd or -rRp when I want to inspect things. -d tells you why things happen and -p shows you what rules and variables are defined.

My bugbear with make (and even the better remake) is finding out why things are not happening. "No rule to make target" is such an uninformative error message. Could it not identify rules which would make the target were they not failing?

Msbuild (as used in Visual Studio and dotnet core) has an amazing implementation of this: https://msbuildlog.com/

Not only can you see what was rebuilt and why, you can see all the input into the rules at all the stages of the build, conveniently hyperlinked into the master rule files provided in the SDK.

i think one of the big problems is that when projects get unwieldly with code generation and external systems, build steps often have unintended side effects which then trigger other build rules unexpectedly.

i remember reading about some wild system on here that exposes the entire source tree via fuse so that it can enforce rules about what changes and what doesn't for each step. wish i could remember the name...

edit: ah, yes. tup. https://gittup.org/tup/

edit 2: see tup graph: https://gittup.org/tup/ex_dependencies.html

FYI you can get this from Nix by giving a .drv file to 'nix-store --query --tree' (a .drv file defines how to build something; it's like an individual Make rule (after all variables, etc. have been evaluated)).

You can get a .drv file using 'nix-instantiate' (or 'nix repl'), e.g.

    $ nix-store --query --tree "$(nix-instantiate -E '(import <nixpkgs> {}).bash')"
    /nix/store/378a0jpkxiwzyk6jj6w2y2qdpn8790nc-bash-4.4-p23.drv
    +---/nix/store/j0bqs7l21myz0llnjp9wq73m4nfarzg1-bootstrap-tools.drv
    |   +---/nix/store/71d9qf26s1nzrp34cqw1hh0yrx0h1kbx-cpio.drv
    |   +---/nix/store/7jl6nhfsc0mchccg5w40rfm4zfshpsag-bzip2.drv
    |   +---/nix/store/bci5x35014bbpc87x7ds2n2ffh5syns1-sh.drv
    |   +---/nix/store/fnn1nmssk7fsfn83ik66cxddqxzvnwx6-bootstrap-tools.cpio.bz2.drv
    |   +---/nix/store/p1bhqmlpf6vr9bc5mb99w29mvllcqmsm-mkdir.drv
    |   +---/nix/store/spcnl3whhm392kir1snaar4wlcawr3yq-unpack-bootstrap-tools.sh
    +---/nix/store/04npl9bjcjkpqgwx1s94azbimfgr531v-bash44-012.drv
    |   +---/nix/store/j0bqs7l21myz0llnjp9wq73m4nfarzg1-bootstrap-tools.drv [...]
    |   +---/nix/store/z6aidzdk3yl2k92g8qj8xgzr8glp0vjr-bootstrap-stage0-stdenv-darwin.drv
    |   |   +---/nix/store/1i5y55x4b4m9qkx5dqbmr1r6bvrqbanw-multiple-outputs.sh
    |   |   +---/nix/store/bnj8d7mvbkg3vdb07yz74yhl3g107qq5-patch-shebangs.sh
    |   |   +---/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh
    ...

It can also output other formats, like GraphML and GraphViz. Note that querying a build output will get its runtime dependencies instead of build-time dependencies.

https://man.archlinux.org/man/community/nix/nix-store.1.en#S...

    ninja -d explain

?

Ninja's "-d explain" is next to useless in a large build: It says what's dirty but it's hard to relate which dirty input caused which output to be re-built. At the beginning of your build you get pages & pages of "these things are dirty" and then it starts the actual build. Also, this "explain" output is printed before ninja has realised that a lot of those things aren't actually dirty (thanks to "restat").

I've been using this in my build system to make the explain output slightly more useful: https://github.com/ninja-build/ninja/pull/2067

It's just a quick hack that took me half a day (most of which was remembering how to C++ after 8 years...) but I think it makes "-d explain" much more useful.

Really, the "explain" implementation could do with an overhaul. I'd love ninja to serialise the build graph to disk (or maybe just the subset of the graph that actually needed rebuilding). This could be augmented with the reason why each rule was run, and possibly even with the command output for each rule. Then the existing ninja tools like "graph", "browse", and "query" could optionally use this graph instead of the "build.ninja" file, so that you can inspect what happened the last time you ran "ninja".

Realistically I'll never get around to implementing this, and my pull request above gets me close enough.

I don't understand why they can't all have interactive debugging either.

Reply on news.ycombinator.com