Hacker News Reader

Comment by lairv | Hacker News Reader

I just spent 3 days to statically link a third party dependency to my C++ project, in a way that works with linux/windows/macos. The package wasn't available on conan/vcpkg, it was just a github repo with a weird combination of Makefile/cmake file. While I'm far from a cmake/cpp expert, this is a non-issue with most modern languages: you just pip install, cargo add, npm install, go get etc. You can read front-to-cover books about every details of C++ semantics and it'll still be a PITA to work on real world projects

View on HN

The javascript toolchain for dependencies and build is as arcane as the sloppiest Makefile.

It's amazing that we ended up with the javascript equivalente of autotools.

yeah, dealing with npm really made me feel like the best part of autotools

Created an account just to say: I am a 10-year JavaScript veteran who knows the ecosystem at least as well as any sane person could possibly be expected to, and this is 100% correct. When I tell people that JS build tooling is the second-worst headache to C/C++ build tooling, they laugh, but stick a toe outside the happy path of `npm create <some-framework-project> && npm start` and you will know what pain feels like.

I did get some similar vibes trying to work with Gradle and Maven for a project a while back, but I don't really live in that world so it's hard to say whether my experience was typical or just symptomatic of my inexperience.

And it'll continue to get worse because JS devs love coming up with their own abstractions over everything and piling complexity. You don't write JS, you write TS. You don't write HTML, you write JSX. You don't write CSS, you write Tailwind. And of course, React has its own compiler as of recently. Now you have to figure out in what order to run these parsers, bundle everything together including your hundreds of dependencies, minimize, obfuscate, tree shake, and whatever else fairy dust magic you want to sprinkle on top. Meanwhile, the default build tool of choice changes about once every 2 years, and now you also have a choice of runtime - node vs bun vs deno. Can't wait to see what the next revolutionary idea would be to contribute to this madness.

There are those of us out there that have a beautiful experience with JavaScript. Vanilla JS, web components, a lightweight rendering library like lit-html, class props as reactive state...

NodeJS + express + vanilla web components is the most graceful and productive stack I've ever used in 30+ years of development.

You forgot to mention the 3 or 4 different TypeScript compilers!

I do have to admit at least some of the pain is self-inflicted on my part. I _want_ those nice abstractions, so I will write a damn Rollup or Babel plugin if it means I can use Sass stylesheets in my TypeScript+JSX components that compile to vanilla Custom Elements (so I don't have to depend on a huge framework runtime).

But as nice as the Babel/Rollup/Vite plugin APIs are, when you start doing stuff like that you then have to deal with all the deep-in-the-weeds bullshit that comes with it, like the fact that Webpack and Rollup and Node.js all have subtly different (and mostly poorly documented) module resolution rules and that they all differ wildly from the official ES-module spec that the standards committee finally shipped like 20 years too late, so trying to get your unit-testing framework and your bundle toolchain and your browser to all agree about how to digest the mess you've made becomes this giant clusterfucktastrophe that makes you question every life decision and formative event that led you to the moment you thought this might be a neat idea.

But yeah just like build a React app or whatever and you'll probably be fine.

Good luck trying to use these tools with anything other than the blessed path of node on the backend.

The JS ecosystem absolutely has the problem of "too many compilers", and of course as I'm typing this there's someone out there writing a "one compiler to unify them all".

What was particularly difficult about it? Sometimes modifying someone else's code to build statically or on another platform is tricky. Linking is generally just include dir, lib path, and lib though.

The library was duckdb (https://duckdb.org/docs/installation/), and here are some of the issues I ran into:

- trying to compile against the prebuilt duckdb_static.so file got me a ton of undefined reference error, I asked about this on their discord and it seems to be a knwon issue, there are additional dependencies that are not part of the release so I had to build it myself

- the library uses CMake but it contains a Makefile and they recommend using that to build it

- It seems like on windows, if you create a dynamic library you have to add __declspec(dllexport) before each function. DuckDB has a #ifdef DUCKDB_STATIC_BUILD to toggle/disable it which isn't documented anywhere, you have to read the code

I'm sure someone will tell me that this is very standard and shouldn't take me 3 days to figure it out, but with basic knowledge of cmake and build systems that's a bit of a pain, and there's no other language where you have to do that

CMake, the defacto if not standard build system, when correctly setup, does all this for you.

Your argument is along the lines of "the maintainer of X rust package didn't setup cargo correctly and it's impossible to include in my project, rust is a horrible language". You can absolutely make an argument that the lack of a standard build system is a pain point, and well it's C++ you get to be a special snowflake and use a makefile instead of CMake but then the onus is on you to provide correct documentation.

In the case of duckdb they provide proper packages for pip, cargo, npm, go get, and the only one that seems like a PITA to use is the C/C++ dependencies, even though the package is written is C++, so I think this tells something about C++ dependency management

You seem to be agreeing with me then, weird take to do it while sounding like you disagree...

As I said, you can make the argument C++ build systems are no good. However the argument originally made is that C++ the language caused these issues when in reality the package maintainer hasn't put in the effort to properly document and fix known issues in their chosen build system for C++.

The fact that they in fact have built out proper packages for pip, cargo, npm and go tells me they have the necessary expertise on the team and made an active choice to not do the same for C++. Create an issue with the maintainer.

The argument originally made was "You can read front-to-cover books about every details of C++ semantics and it'll still be a PITA to work on real world projects"

Which is not disagreeing with you just because you are explaining/disputing the reason it was a PITA.

That statment was qualified by:

> While I'm far from a cmake/cpp expert, this is a non-issue with most modern languages: you just pip install, cargo add, npm install, go get etc.

Which I also find fundamentally wrong because most real world projects have a C/C++ dependency which needs to be built and is built from one of those languages build systems by calling into cmake / autotools + make.

Just because you called apt-get or any other way to pull dependencies doesn't remove the fact that they need to be built and likely involve a C compiler.

Seeing as it's Christmas Eve and I has nothing better to do...

Duckdb own documentation says the C++ API is internal and you should use the C API[1]

The makefile also has a 'bundle-library' option which seems to be exactly what you were looking for, it generates a statically linked library which is what the Golang package is using, probably others too but that's the one I checked first.

This is purely a documentation problem, the build system does what is needed it seems. Create an issue, or better yet, add the necessary documentation...

1. https://duckdb.org/docs/api/cpp

I used the C api but I did miss the 'bundle-library' option, will look into that

I definitely don't blame you for missing it.

I found it by looking at the go package because they claimed they are statically linking there.

> the maintainer of X rust package didn't setup cargo correctly and it's impossible to include in my project

That basically never happens, which is the point.

I have seen way more projects with very broken CMake integration than rust packages with broken cargo setup.

CMake is a pain.

> trying to compile against the prebuilt duckdb_static.so

Typically you statically link against .a object library. .so are shared objects intended for dynamical linking only. Does duckdb provide something like duckdb.a ?

That was a typo, the file was duckdb_static.a

Ha, I used duckdb once and ran into a memory corruption issue through normal usage. It was a known issue also surfaced by a fuzzer. One of the devs referred to the fuzzer as the fusser in the issue about it and I stopped using the project immediately. Best of luck with that one!

So you stopped using a tool because someone misspelled a word?

I stopped using it because it was in a memory unsafe language, had readily triggered crashes, and because a developer seemed to security as a chore.

But to flip this around, let's say you want to use this third party in some other "modern" language how would this become better?. Clearly if someone hadn't done the work to package in vcpkg/conan you can expect it wouldn't exist in pip/cargo/npm either. So if you had a bad time in C++, you would have a dramatically worse time in other languages. Goes to show what everyone else is saying here, there are parts of historic C++ which are in very bad shape particularly the build story. However, if it's in vcpkg it is exactly as easy to install as the other ones.

> But to flip this around, let's say you want to use this third party in some other "modern" language how would this become better?. Clearly if someone hadn't done the work to package in vcpkg/conan you can expect it wouldn't exist in pip/cargo/npm either. So if you had a bad time in C++, you would have a dramatically worse time in other languages.

Err, no?

In case of Rust usually there is really no work that's necessary to package it and upload. You just type `cargo publish` and you're done (after filling out metadata in Cargo.toml). Even if someone wouldn't upload it to crates.io as a Rust crate you can just add a single line to your Cargo.toml:

    package_name = { git = "https://github.com/user/repo", rev = "revision" }

and you're done, you can use it as a dependency.

I think you moved the goalposts. I can configure a CMake file to pull a GitHub repo, which seems similar enough.

I think the difference is that you can pip install a github url and it will work, it will even work if the upstream has done nothing to package it.

I think the issue could be broadly defined as how difficult it is how difficult it is to take someone else's code and use it as part of your project. For C/C++ the answer can be completely trivial to a massive pain in the ass.

Oh I dunno. With cmake it really is pretty straight forward. You can pull from VCS, a tarball, even use a git submodule.

If it's another Cmake project, that's the happy path. Easy peasy. If it's not you still have the ability to run ./configure && make or evoke whatever incantation is in the dependency's requirements. I'm not the hugest fan of cmake, but I've worked with it quite a bit and it's a much more pleasant story than other languages I've used a lot like python, JavaScript, OCaml (which I love), etc.

I do find rust and go dependency management a little simpler due to the prescribed nature of them, but once you get off the happy path, the flexibility of C land is tough to beat. It sort of matches the language stories themselves; you have more control and have to do a bit more yourself. It's all about trade offs.

How often do folks venture off the happy path with Rust dependencies? Personally I’ve never once found myself in that situation.

I've done it before to integrate rust into polyglot build systems. There's a surprisingly long history of build systems trying and failing to implement rust builds without the "happy path" of simply wrapping cargo. As far as I know no one's ever succeeded.

The main difference being that Cargo is both for distributing and building locally AND it’s the blessed way of building all Rust projects. This means that as long as you have a copy of the codebase, it’s going to be using Cargo & you can add it with Cargo regardless of it being published formally to crates.io (the copy can even be remote since you can build from a git reference directly). Python packaging requires more work although there’s typically no build step so you can probably just __import__ the path and move on. JS is closer to Rust in that package.json is the standard way to manage components and you can import them even if they’re not published via the npm registry.

Pretending the situation isn’t very different for C++ is completely ignoring the standard committee’s complete abdication of standardizing a build system (which by the way if they’re not going to do, why are they bothering to standardize anything?)

Reply on news.ycombinator.com