Hacker News Reader

Comment by chris_overseas | Hacker News Reader

Agreed, and the small stdlib is one of the main reasons for this problem. I understand the reasoning why it's small, but I wish more people would acknowledge the (IMHO at least) rather large downside this brings, rather than just painting it as a strictly positive thing. The pain of dealing with a huge tree of dependencies, all of different qualities and moving at different trajectories, is very real. I've spent a large part of the last couple of days fighting exactly this in a Rust codebase, which is hugely frustrating.

View on HN

What is the reason to keep it small? Genuinely interested, I actually don't understand.

Embedded systems maybe?

AFAIK: Rust compiles to machine code. Even if the stdlib would be 600 mb, If you have 3 lines of Code your programm would be microscopically small.

(I've been on libs-api, and libs before that, for 10 years now.)

API Stability. When the standard library APIs were initially designed, "the Python standard library is where packages go to die" was very much on our minds. We specifically saw ourselves as enabled to have a small standard library because of tooling like Cargo.

There are no plans for Rust 2.0. So any change we merge into std is, effectively, something we have to live with for approximately forever. (With some pedantic exceptions over edition boundaries that don't change my overall point.)

Nuance is nearly dead on the Internet, but I'll say that I think designing robust and lasting APIs is a lot harder in Rust than it is in Go. Rust has a lot more expressiveness (which I do not cite as an unmitigated good), and the culture is more heavily focused on zero-overhead abstractions (which I similarly do not cite as an unmitigated good). That means the "right" API can be very difficult to find without evolution via breaking changes. But the standard library cannot, generally speaking, make breaking changes. Crates can.

I would suggest not reading the OP as a black-and-white position. But rather, a plea to change how we balance the pros and cons of dependencies in the Rust ecosystem.

I can understand not wanting to add SMTP or CGI to the stdlib. But a lot of common POSIX functionality (which is sometimes a single syscall away) is missing too.

A lot of common POSIX functionality is not missing though. I was able to write ripgrep, for example, by almost entirely sticking to the standard library. (I think the only place I reach out to `libc` directly is to get the hostname of the current system for rendering hyperlinks in your terminal.)

We also came at the standard library with a cross platform mentality that included non-POSIX platforms like Windows. You want to be careful not to design APIs that are too specific to POSIX. So it falls under the same reasoning I explained above: everything that goes into std is treated as if it will be there forever. So when we add things to std, we absolutely consider whether the risk of us getting the API wrong is outweighed by the benefit of the API being in std in the first place. And we absolutely factor "the ease of using crates via Cargo" into this calculus.

I peeked at the code for gethostname in ripgrep, and it's nice and straightforward.

Much like op said here; we have a culture of "don't write unsafe code under any circumstance", and we then pull in a dependency tree for a single function that's relatively safe to contain. It solves the problem quickly, but at a higher price.

BTW, thanks for ripgrep. I don't actually use it, but I've read through different portions of the code over recent months and it's some very clean and easy to understand code. Definitely a good influence.

I don't think you should treat unsafe code as that level of toxic. It's necessary when interfacing with with system APIs. The important part is that you try to have safe wrappers around the unsafe calls and that you document why the way you're using them is safe.

I've been using ripgrep for years! Thanks a lot for that!

The three main reasons I see being given are:

- backward compatbility: a big std lib increases the risk of incompatible changes and the cost of long term support

- pushing developers to be mindful of minimal systems: a sort of unrelated example is how a lot of node library use the 'fs' module just because it is there creating a huge pain point for browser bundling. If the stdlib did not have a fs module this would happen a lot less

- a desire to let the community work it out and decide the best API/implementations before blessing a specific library as Standard.

In my opinion a dynamic set of curated library with significantly shorted backward compatibility guarantees is the best of both worlds.

Other reasons also include:

- less burden on the stdlib maintainers (which are already overworked!)

- faster iteration on those libraries, since you don't need to wait a new release of the compiler to get updates for those libraries (which would take at least 12-16 weeks depending on when the PR is merged)

One risk with a bigger standard library is that you'll do an imperfect job of it, then you'll be stuck maintaining it forever for compatibility reasons.

For example, Java developers can choose to represent time with Unix milliseconds, java.util.Date, java.util.Calendar, Joda-Time or java.time.Instant

It’s really just Date and Instant. Joda-Time isn’t part of the standard library. And if you’re listing Calendar, you might as well also list ZonedDateTime, OffsetDateTime, and LocalDateTime, not to mention stuff like java.sql.Date.

In reality, there’s just one old API and one new API, similar to the old collection classes (HashTable, Vector, etc.) and the newer JCF ones.

In addition to the points burntsushi gave in the sibling comment, I'd also add that keeping the standard library small and putting other well-scoped stuff like tegex, rand, etc. in dependencies also can reduce the burden of releases a lot. If some a bug gets found in a library that's not std, a new release can get pushed out pretty quickly. If a bug gets found in std, an entire new toolchain version needs to be published. That's not to say that this wouldn't be done for critical bugs, but when Rust already has releases on a six-week cadence, it's not crazy to try to reduce the need for additional releases on top of that.

This probably isn't as important as the stability concerns, but I think it still helps tilt the argument in favor of a small std at least a little.

Reply on news.ycombinator.com