Hacker News Reader

Comment by progval | Hacker News Reader

Concrete example: the Apache Arrow format is designed to allow passing arrays across languages and libraries without conversion; so many public functions/methods (and FFIs) will have Arrow arrays in their signature.

However, the Rust Arrow library bumps its major version every few months because it changes methods and auxiliary types, without changing the memory layout of arrays. But this means you can't have two dependencies (eg. an ORC parser and a CSV serializer) that depend on different versions of Arrow if you want to pass arrays from one dependency's functions to the other's.

And vendoring like you mention won't help, because the Rust compiler wouldn't recognize both vendored Arrow crates as compatible.

View on HN

Use a single version of the arrow library, and build everything against that one. I don't see where the problem is.

I don't use Rust or third-party build or packaging systems -- I usually recommend that people don't do so either, but I understand that cargo is part of the appeal of Rust.

I'd say just build whatever you need to do, this way you won't be limited by arbitrary restrictions others have decided.

I can't always do that because dependency A depends dependency B that depends on arrow 52, and dependency C depends on arrow 53.

And sure, I can soft-fork dependency B to add support for arrow 53, and dependency A to depend on my soft-fork of dependency B, but it quickly becomes messy. Especially when in parallel I send unrelated patches to A and/or B.

your problem is caring about what other people have decided. Surely metadata is irrelevant and can be changed/overridden.

You onboard a version and make sure everything you need can build against it, patching software as needed.

Security patches say no, doesn’t work that way.

Sounds like a team of misanthropes. How can you be that blind to DevEx?

For the best developer experience, you need to write custom tools.

Eh. Software is full of cliches and idioms. I shouldn’t be writing my own logging library and I sure as fuck shouldn’t be writing my own security library.

Also don’t write your own billing system unless you’re a finance company.

Reply on news.ycombinator.com