Why people think it's a good idea for the build systems to reach out and grab code from the internet is beyond me.
Supporting distribution for all the different kinds of Linux package mangers alone isn't easy, and you take in how macOS and Windows do things, I'm not even sure where you start.
It's kind of sad, but there's multiple walled gardens in play here. I mean, Java's been around for 25 years, and it's just starting to get tooling for making OS installers for applications (jpackage).
So, programming toolchains just invent package management that is easy for everyone to use. Everyone needs a problem solved today for multiple platforms, and it's not like OSes are working together to make this easy.
Personally, I've come to prefer vendored dependencies with the sources, and building scripts around vendoring, separating the build from dependency distribution. But not a lot of programmers are comfortable with this. It's not a common practice, and the most popular version control system (git) isn't very great with binary management.
Do you manually install them on your local host? That's not reproducible and doesn't support multiple versions for different projects.
Do you use an uber build system like Yocto to wrap your CMake build, and let it pull the packages?
Do you manually copy all the external deps into your own project repo, and then manually keep them updated with the latest changes?
Now there do exist "uber build systems" that are written in cmake. I'm fine with those existing (though I would probably not chose cmake to write it in, that is the author's choice). However when building a project you should not force a method to get dependencies on your users. The package management is a system level concern and you should not mess with it.
Note that I just put a lot of hate on systems like Cargo, pip, or hackage. Or google's pull everything into the one repository. Dependencies belong to tools like apt, rpm, yocto, ,snap, conan (when not building packages to install) - Microsoft/Apple are deficient for not providing them. Your program should just attempt to build with what it is given, not try to subvert my system - whatever it is. Yes I know it makes your life easier to not have to deal with more than one [possibly buggy] version of a dependency, but it makes my life worse.
This makes anything approaching a hermetic/deterministic/reproducible build basically impossible. That's not good. How do I get consistent builds across systems if I have to hope that some (set of, if I'm building on multiple OSes) external dependencies give me the correct version of the package I'm looking for, especially since many of the tools you listed don't, for example, support multiple simultaneous versions of a particular package.
I'm also confused by precisely what you mean by package management. Tools like pip cargo and bazel etc. Don't manage compiled binaries, but uncompiled sources. They deal with code dependencies, not dynamic library dependencies.
Installing a source file via apt doesn't make a whole lot of sense.
Have a script which generates build project/solution/script files for the actual build.
These scripts can use whatever tools your organisation is most comfortable with, up to and including a mono repo.
Either using a package manager, or doing it "manually" (like you suggest) – a process which can of course also be automated outside of the build system.
> Do you manually install them on your local host? That's not reproducible and doesn't support multiple versions for different projects.
Of course that's reproducible and supports multiple versions. Any build system worth its salt can be directed to look for dependencies at specific locations. Then multiple versions of things can be installed at multiple locations. This is literally how things have been done for ages.
> Do you manually copy all the external deps into your own project repo, and then manually keep them updated with the latest changes?
I abhor bundling dependencies, so I'm probably not the right person to ask, but supposing that you do want to bundle dependencies: yes (a process that can be scripted just as easily as it can be handed off to the build system!)
Being able to ship software, especially across many platforms is much more important than maintaining some sort of purist separation between build system and package manager. Your view of affairs, after being dominant for decades, is finally receding into the background, and the world is a better place for it.
The separation of concerns is of far greater value than just purism. It can ease cross-compilation, it can help building in offline environments, it can make dependency customization far easier, it can make swapping out compilers much easier, etc.
These are all things that have been valuable in the past, and there's little reason to think that they aren't still valuable and will be valuable in the future.
Cross-compiles are part of the fundamental model of cargo (target vs host dependencies).
cargo can be used in a completely offline manner easily, through either caching dependencies in an online step or vendoring them (or just not using dependencies).
Cargo has pretty good support for dependency patching.
The compiler can be swapped out through an environment variable, RUSTC_WRAPPER.
I agree that all of those things are valuable! The cargo developers think so too which is why cargo supports all of them.
I suspect the future is more in making tools like cargo support all the use cases people are concerned about, than trying to find an orthogonality where one doesn't exist.
You don't have to do that. If you separate the task of building from the task of managing software, you can mix and match solutions for each separate task as you wish.
> And I've seen makefiles with "sudo apt install libqt5core libxml2-dev ..." in them and I didn't like that.
That's obviously terrible too.
That is the saner option among the ones that you mention.
The sanest, of course, is to not update your dependencies unless really necessary, to avoid introducing new bugs.
So instead you live with all the existing (and unknown) bugs and security issues?
And when you finally do update those frozen dependencies it becomes a nightmare because of all the drift, not just in that package but also its dependencies.
You should be updating regularly; if you're concerned about stability then stay a release or two behind latest but that's not guaranteed to more stable than latest.
Too often I see people take the approach you're espousing to only suffer more later when they are forced to take an update, and then they become so scared of future updates that they never update again, manually backporting patches, and creating a nightmare legacy application.
My experience over the 40 or so packages that I regularly use is that updates always have less bugs that old versions though. Even for my system, I build most of the software I use from git head and it just works better than whatever's in Ubuntu or Debian on average
Also, separation of concerns is a good thing IMO.
"Separation of concerns" is ill-defined — you can use cargo as just a package manager, just a build system, or both. Most people use it as both but there are definitely places (e.g. bigcos) that use just one of the aspects of cargo.
What is missing is a modern cross-language, cross-platform package manager, I agree. But there are really good reasons for that, chief among them being that I don't think it's possible to define the scope of such a thing in a coherent fashion.
Indeed, and I'm still highly skeptical. Learning Rust has been an amazingly rare sequence of "wow that's perfect" uttered from my mouth – except for Cargo. I couldn't disable the networking functionality of Cargo fast enough!
Eh. CMake is a build system. It supports multiple build tools, but it's not like one can just tell CMake "here's some literal Make code, please put it into the generated files". The generated builds are still restricted to the semantics that CMake itself provides via its interface (e.g., no dyndep support in `add_custom_command` because only the Ninja generator supports that reasonably well; it's possible with Make, but I don't know how easy it would be to make it work for arbitrary custom commands).
One cannot use CMake to write arbitrary Makefiles using all the fancy features available through macro expansion, rule templates, etc. All you have are CMake's semantics that can then be executed by a tool that implements POSIX makefile execution behaviors.
A (probably poor) analogy is a network resource. It can be provided over HTTP, Gopher, or whatever. The semantics of the resource are the same, but the transport mechanism is just how it interacts with the rest of the world.
Many times most of the dependencies are available through the package manager of ones system.
A good way in my opinion is just to provide a clear list of the dependencies and make the build system complain if it's not there so one can install it (if didn't bother manually to check that the listed dependencies are present).
Of course there are very large and complex projects which may need more sophisticated solutions - Perhaps some sort code notarisation would serve well? Yet even there the dependency management should be clearly separated from the build system and it should prioritize using system provided packages when possible.
There are probably other considerations and implications too. Like preference of static linking over dynamic etc..
How terrible an idea it is depends on who's providing it, same as most other software.
There are some specific pitfalls with curl|bash:
(A) the web server may serve different code for curl's user-agent than what you audited by clicking the link in a browser, leading to a false sense of security; (B) if the network transfer is interrupted, bash will partially execute an incomplete script; and (C) the recurring prevalence of high-profile bugs like https://github.com/valvesoftware/steam-for-linux/issues/3671
You can mitigate (B) by wrapping the whole script in a function, and you can mitigate (C) with `set -euo pipefail` and `trap`, but that doesn't seem to be the kind of bash scripts that people write in practice.
MSI (and deb/rpm) definitely can arbitrary bad things at install time too, but at least their built-in file extraction features are entirely declarative, and they always have an entrypoint for the system uninstaller.