Hacker News Reader

Comment by jtrueb | Hacker News Reader

The reason for not choosing Rust still doesn't make any sense to me. If you don’t want to OOM, need correctness, are following the power of ten (where you aren’t allocating anyways), I don’t see the conflict or harm of additional enforced correctness.

Also, Rust does support checked arithmetic and has stable toolchains.

View on HN

Several types of correctness are more difficult to enforce in Rust than in some other languages due to its relatively weak compile-time facilities. Modern database engines don't allocate memory at runtime, aren't multithreaded, they explicitly schedule ownership, etc. They also use the types of data structures and patterns that give the borrow checker fits. Rust's correctness capabilities are relatively less valuable in this context.

Extensive compile-time capabilities that allow you to verify many other types of correctness add enormous value when building database engines. This is a weak part of Rust's story.

Can you elaborate a little more on what structures and patterns those are? I have built some database like things in rust (lots of zero copy and slice shenanigans, and heavily multi-threaded) and while it was tricky in some spots I haven't run into anything I have had to substantially compromise on.

Your use cases are likely more complex, so I'm curious what I should be looking out for.

My understanding from reading other blogs on TigerBeetle (and the Power of Ten rule) is that it's not that they aren't allocating at all. It's all static allocation up front. Zig makes these far easier to manage with its use of Allocators. Rust wants everything to be RAII, tons of little allocations who's lifetimes are managed by the borrow checker. You can use other patterns in Rust of course but you're fighting the borrow checker.

Zig gives you a lot of tools to enforce correctness in simple and straightforward ways, where as Rust comes with a lot of complexity. TigerBeetle isn't the first project to talk about this, Richard Feldman also points out similar advantages to Zig over Rust as the reasoning for the Roc compilers rewrite from Rust to Zig.

There's a change in the tradeoffs in the above scenario:

- you still get extra benefit from Rust, but the magnitude of the benefit is reduced (e.g., no UAF without F).

- you still get extra drawbacks from Rust, but the magnitude of drawbacks is increased, as Rust generally punishes you for not allocating (boxing is a common escape hatch to avoid complex lifetimes).

Just how much tradeoff is shifted is hard to qualify unambiguously, but, from my PoV (Rust since 2015, TB/Zig since late 2022), Zig was and is the right choice in this context.

I mainly use Rust in embedded now. I don’t always rely on encoding all of the correctness in the Rust type system. To a degree all the old ways of enforcing correctness are still in play, I am just choosing when to take use idiomatic Rust or escape hatch out via shim to C-style Rust. It reminds me quite a bit of how C and C++ shops require another layer of macros or templates be used for containers, resources, etc.

The build time of Zig seems like the most desirable piece worth deciding over. Developer time is money, but it isn’t weird to have multi-hour build times in a mature project either C, C++, or Rust. The correctness suite is a bigger time sink than the build though. When building a database, you could drive the build time to 0 and still have hours in CI.

Out of interest, did you read the two posts [0][1] linked in there by matklad, creator of rust-analyzer as well as IntelliJ Rust, on our team?

Suffice to say, we know the intrusive memory and comptime patterns we use in our code base, and they wouldn't be as natural to express in a language other than Zig.

Rust is a great language, but Zig made more sense for what I wanted to create in TigerBeetle.

[0] https://matklad.github.io/2023/03/26/zig-and-rust.html

[1] https://lobste.rs/s/uhtjdz/rust_vs_zig_reality_somewhat_frie...

Yeah, I think BDFL wants to use Zig. I understand that it is nice for Zig to feel more like C, and that can be fun. If the toolchain is so far away from being mature, how long will it take the database to be mature?

Since previous comment was edited. I would clarify that I don’t doubt the engineering capabilities, just the timeline. A from scratch database in _established_ toolchains take 5-10 years. The Zig toolchain also is going to be evolving in the same timeframe or longer. The codegen, linking, architecture specific bugs etc. Isn’t it double the effort to bring to bear in the market?

You're right! In the past, distributed databases written without Deterministic Simulation Testing (DST) would typically take around ~10 years (~5 years for the consensus protocol and ~5 years for the storage engine). And then not without bugs.

However, and we write about this also in the post, but TigerStyle and DST enabled us to ship TigerBeetle to production in 3.5 years, in less time, to a higher standard, and to be the first distributed database with an explicit storage fault model (Kyle Kingsbury added new storage fault injectors to Jepsen) solving Protocol-Aware Recovery.

Our Jepsen audit is here (also linked in the post): https://jepsen.io/analyses/tigerbeetle-0.16.11

For more on TigerStyle as a methodology, hope you enjoy this talk: https://www.youtube.com/watch?v=w3WYdYyjek4

I was on a team with a similar timeline with C++ (4 year). All the language and toolchain difficulties came after shipping. Meeting new customer needs meant shifting from greenfield to brownfield engineering. We were chasing down strange platform and provider behaviors. Adding features while maintaining performance and correctness, meant relying on knowledge of tools available in the broader community. Solutions for build issues came through a combination of in-house effort and industry partners with related experience. Having two stable compilers (gcc and clang) was super helpful.

Reply on news.ycombinator.com