Hacker News Reader

Comment by nicoburns | Hacker News Reader

Part of this is just crates being broken up more in Rust. For example the `http` crate only contains trait (interface) definitions. They break down like so:

Platform integration: libc, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys, fuchsia-zircon, fuchsia-zircon-sys, kernel32-sys, redox_syscall

Primitive algorithms: itoa, memchr, unicode-xid

Proc macro / pinning utilities: proc-macro2, autocfg, cfg-if, lazy_static, quote, syn, pin-project, pin-project-internal, pin-project-lite, pin-utils

Data structures: bitflags, bytes, fnv, hashbrown, indexmap, slab

Core Rust asyncio crates: mio, miow, iovec, tokio, tokio-util, futures-channel, futures-core, futures-sink, futures-task, futures-util,

Logging: log, tracing, tracing-core

The following are effectively sub-crates of the project: http, http-body, httparse, httpdate, tower-service, h2

Not sure what these are for: net2, socket2, try-lock, want

View on HN

Honestly it still seems like a lot this way.

If those platform crates are just backends for libc (or similar to libc), why aren't they all folded in a single project? Having them as separate crates open the gate for supply-chain attacks without really allowing greater control or expressiveness. You are always going to pull all of them in, you are unlikely to ever use them directly, and they have no more impact on compilation than features.

Having to use proc-macro2+quote+syn for macros feels wrong, considering it's a language feature.

Having to pull in 4 crates for pinning also seems wrong. This could easily be a single utility crate, and if you really need all this cruft to use Pin (a language feature) most of this should probably be in std.

The async I/O feels like definite bloat. Not only is that a lot of futures-* crates, but I know from first-hand experience that those tend to implement multiple versions of some primitives (like streams) that are incompatible.

> If those platform crates are just backends for libc (or similar to libc),

They're not, they're to the specific platform's APIs. The libc crate is a package that targets libc on every platform.

> Having to use proc-macro2+quote+syn for macros feels wrong, considering it's a language feature.

In order to ship procedural macros as a feature, they're pretty minimal. There's a tradeoff here; others could be chosen, but weren't for good reasons.

> Having to pull in 4 crates for pinning also seems wrong.

This is covered by https://news.ycombinator.com/item?id=24730280; that is, it's a transitive dependency issue.

> The async I/O feels like definite bloat.

Really depends, network calls are a textbook use-case for async I/O. And there's so many futures crates specifically so that you can only depend on the bits you need.

Why does hyper pull in hashbrown? Isn't it identical to std::collections::HashMap?

I imagine that it's there because it is older than that happening. HashBrown's README says

> Since Rust 1.36, this is now the HashMap implementation for the Rust standard library. However you may still want to use this crate instead since it works in environments without std, such as embedded systems and kernels.

I don't know if that use case is important to Hyper or not.

The dependency goes hyper -> h2 -> indexmap -> hashbrown, and indexmap is built on hashbrown::raw::RawTable.

Looks like the indexmap -> hashbrown dependency was added recently: https://github.com/bluss/indexmap/pull/131

Thinking about crates this way is a revelation to me. Is there any tooling to make analyzing dependencies as you did easier?

That. And then we become responsible for yet more possible vulns in all those deps. Current number of cURL deps: 14

I wish it relied only on OpenSSL.

Not depependin on OpenSSL is kind of the point of the original post if I understand it correctly.

Also I find the dependency on OpenSSL one major pain in my Rust projects. When you want to build a statically linked binary you need to supply a statically built OpenSSL and if your distro doesn't come with one (like Ubuntu) you are on your own. Yes, there is a Docker container that comes with all the prerequisites but I think that's a bit heavy for my purposes.

I wish there was a single switch in Cargo.toml and every dependency would automagically use rustls.

Containers is becoming the way to build stuff, partly for that reason.

I think any dependency adds a level of burden, but some things are better delegated to library. I think crypto is a good case, btw I think OpenSSL is not the only one lib for TLS with curl.

> Containers is becoming the way to build stuff, partly for that reason.

This sits wrong with me, but thinking about it: I rewrote the sentence about containers in my comment three times before posting it and it still doesn't sound compelling. Maybe you have a point here.

You dont have to use OpenSSL. For example Windows already includes Schannel, so you can build cURL with Schannel and avoid OpenSSL:

https://daniel.haxx.se/blog/wp-content/uploads/2020/09/curl-...

So this gets you also different behaviour that may be what you want, or may not, depending.

Specifically if you use SChannel, you get the CA roots from Microsoft's CA Root programme, whereas ordinarily you'll end up with (some derivative of) the Mozilla CA root programme.

You also get the local policy root overrides. So for example in many corporate networks with a middlebox ensuring employees don't look at porn, the middlebox is trusted according to Windows Group Policy. Now your Curl program works the same way as Internet Explorer does, if the site is trusted in IE then it's trusted in Curl.

On the other hand, this means that the SChannel enabled Curl trusts different things from the Curl on platforms with OpenSSL. Maybe this new setup works "fine" in SChannel Curl, but only when you try from a Linux do you discover that your new site doesn't work at all any more without Microsoft's trust list, which explains the thousands of new tickets filed by (mostly Linux using) customers whose product just mysteriously broke even though it looked fine on your Windows test machine and you've just closed a dozen of those tickets as WORKSFORME...

Https://Www.Github.com/mimoo/dephell

Sorry www.github.com/mimoo/cargo-dephell

There is but for me cargo tree was always sufficient so far.

cargo-crev is one

Reply on news.ycombinator.com