oconnor663

Very cool! Is your work all by hand, or have you been able to automate some of it?

> It's true that Rust UB can only arise from unsafe blocks, but it is not limited to unsafe blocks.

This is correct, and it's hard to teach, and I agree that a lot of folks get it wrong. (Here's my attempt: https://jacko.io/safety_and_soundness.html.) But I think this comment is understating how big of a difference this makes:

1. Rust has a large, powerful safe subset, which includes lots of real-world programs. Unsafe code is an advanced topic, and beginners don't need to learn about it to start getting their work done. Beginners can contribute to big projects without touching the unsafe parts (as you clarified, that means the module privacy boundaries that include unsafe code, not just the unsafe blocks), and reviewers don't need to be paranoid about every line.

2. A lot of real-world unsafe Rust is easy to audit, because you can grep for `unsafe` in a big codebase and zoom right to the parts you need to look at. Again, as you pointed out, those blocks might not be the whole story, and you do need to read what they're doing to see how much code they "infect". But an experienced Rust programmer can audit a well-written codebase in minutes. It's not always that smooth of course, but it's a totally different world that that's even possible.

Apart from runtime/embedded requirements, there's the big question of how you represent what C is doing in other languages that don't have interior pointers and pointer casting. For example, in C I might have a `struct foo*` that aliases the 7th element of a `struct foo[]` array. How do you represent that in Java or Python? I don't think you can use regular objects or regular arrays/lists from either of those languages, because you need assignments through the pointer (of the whole `struct foo`, not just individual field writes) to affect the array. Even worse, in C I might have a `const char*` that aliases the same element and expects every write to affect its bytes. To model all this you'd need some Frankenstein, technically-Turing-complete, giant-bytestring-that-represents-all-of-memory thing that wouldn't really be Java or Python in any meaningful sense, wouldn't be remotely readable or maintainable, and wouldn't be able to interoperate with any existing libraries.

In Rust you presumably do all of that with raw pointers, which leaves you with a big unsafe mess to clean up over time, and I imagine a lot of the hard work of this project is trying to minimize that mess. But at least the mess that you have is recognizably Rust, and incremental cleanup is possible.

I'm guessing, but I'm pretty sure that "admired" is derived from folks who used a language and who want to keep using it.

Note that secure tree hashing requires a distinguisher between the leaves and the parents (to avoid collisions) and ideally another distinguisher between the root and everything else (to avoid extensions). Surprisingly few bespoke tree hashes in the wild get this right.

I'm just guessing, but I imagine the scenario goes like this:

1. Work at a company that runs some ancient version of CentOS or something.

2. The system package manager doesn't have anything newer than Python 3.6.

3. Official binaries from python.org print an inscrutable error about the glibc version when you try to run them.

4. Building Python from source requires some new system dependencies, takes forever, and then produces an even less scrutable error about a missing header file.

5. Googling that error leads to a ./configure flag that works around the issue, which upstream GCC fixed in 2017.

6. Success?

If you haven't seen that error #3 before, or dealt with "manylinux" build environments, you've dodged an entire world of pain. The same goes for scripting installs on Windows, and for the part of that page that's talking about "limiting the CPU instructions that can be used" :')

There are a lot of factors that go into how fast a hash function is, but the case we're showing in the big red chart at https://github.com/BLAKE3-team/BLAKE3 is almost entirely driven by SIMD. It's a huge deal.

> If you are developing in Python or C# you may already know the big cost of async/await: function coloring where sync function can't call async function and vice versa.

The "blue can't call red" property -- the plot climax of the original function coloring article -- only applies to JS. Other languages can call async functions from sync functions using some sort of "block on" or "wait for" API. The function coloring problem is still annoying, but it's not a fundamental limitation like it is in JS.

> it's been running on many VMs without exploding memory for a while now

Most of the security bugs we hear about don't cause random crashes on otherwise healthy machines, because that tends to get them noticed and fixed. It's the ones that require complicated steps to trigger that are really scary. When I look at NTP, I see a service that:

- runs as root

- talks to the network

- doesn't usually authenticate its traffic

- uses a bespoke binary packet format

- almost all network security depends on (for checking cert expiration)

That looks to me like an excellent candidate for a memory-safe reimplementation.

You can put nodes in a Slab or a SlotMap, use keys/indexes for your next pointers, and create multiple interacting (possibly overlapping) lists that way. You have to be careful not to combine nodes from different collections, but that's also sort of true with memory allocators.

I only played with Zig for a weekend, so others will need to chime in, but a few points:

- Zig's comptime is extremely flexible, while Rust's generics are more constrained. It's not clear if Rust will ever get "specialization" for example, but Zig gets that sort of thing for free by making the whole language available at comptime.

- Zig is very explicit about memory allocation, and the standard library doesn't include types like Rust's Vec or String that implicitly talk to a global allocator. If you're writing an application that must not allocate in the render loop / control cycle / etc, that can be a feature.

- Zig doesn't have a borrow checker. If your application plans to use memory management strategies like "a bump allocator that frees everything at the end of a frame" or "a bump allocator that frees everything at the end of a query", it can be painful to express the lifetime restrictions of those allocations in Rust. Zig doesn't get in the way of those strategies, though that also means there's no such thing as "safe code" in the Rust sense.

> Perhaps I am still thinking in C too much.

> I much would prefer the legacy preprocessor macros C...

Holy cow you weren't kidding :)

My wild guess is that it would depend a lot on how much your employer likes you and how they feel about the reason you're in jail.

> If you perform concurrent accesses through multiples processes

How would you do that in safe code? If I understand correctly, the problem you're referring to is the exact reason that mmap APIs in Rust are unsafe: https://docs.rs/memmap2/latest/memmap2/struct.Mmap.html#safe...

This is all deeply hypothetical (and I'm super biased), but I think it's important to distinguish two different questions:

- Is it possible for a private company to make it to space without government money?

- Is it possible to do that while competing with other companies that do get the money?

Observing that "company ___ would not have succeeded without government money" is answering the second question but maybe not the first.

If we could keep running this era of civilization forever, Matrix style, I'm sure that would be true. But how many more generations will there be before AI+biotech lead to us, I don't know, 3D-printing people? The old laws might not have many cycles left to go before the new ones show up...

I've been meaning to turn this into a proper article, but Rust has some really nice advantages once you get comfortable with it, even in programs where performance/embedding isn't important:

- enums

- const-ness without immutable types

- no null

- no iterator invalidation

- no surprising captures in closures/lambdas

- no "spooky action at a distance"

- Cargo

I know what this comment is getting at, but I think the details matter a lot: https://jacko.io/object_soup.html

For comparison "C++ shared_ptr is only safe with tree-shaped structures" would be getting-at-the-truth-but-kinda-weirdly-worded in some similar ways.

To be fair, it's not really the bricks that are so expensive. It's the dirt :)

How do the distro packages work? Are they trying to provide dependencies as pre-built binaries? I didn't know Cargo could consume binaries like that.

Rust can:

- Do the things C and C++ can do.

- Without the memory corruption issues those languages are infamous for.

- With the conveniences you'd expect of any post-internet language. (A library ecosystem that's unified around a standard build system and package manager, an async IO story, UTF-8 strings, etc.)

> Lampson further remarks that to do parallel programming, what you need to do is put all your parallelism into a little box and then have a wizard go write the code in that box. Not much has changed since 2009.

I would like to see a reevaluation of this take with respect to Rust. When we're talking about Rust's safety features, we usually focus on memory safety, because that has the biggest impact on security. But there are lots of memory safe languages, and I think it's actually Rust's thread safety features that are the most unique. (Lots of shared type system machinery between both sets of features.)

Close enough. If the ship had to ignore the flight plan and press to orbit, there's no doubt that it would've succeeded.

I have no idea, but the alcohol content of different wines varies by quite a bit, and probably also changed over time?

Deciding what to do about the version split seems tricky, but on the plus side, it's impressive that they implemented a lot of real hardware before they even froze the extension.

A version of this that seems both easier and less weird would be an AI that listens to you all the time when you're learning a foreign language. Imagine how much faster you could learn, and how much more native you could ultimately get, if you had something that could buzz your watch whenever you said something wrong. And of course you'd calibrate it to understand what level you're at and not spam you constantly. I would love to have something like that, assuming it was voluntary...

> And that is what the C integer types are about, right, about being efficiently represented by the underlying hardware, not their exact bitwidth?

If you have one `short` argument to your function maybe. But if you have a `short[]` array, you probably do care about the memory layout of that array. You might need it to be compatible with some particular data format that you're trying to read/write. Same with a field of a struct, if that struct is used for parsing. A lot of C code does parsing like this.

Counterpoint: The only really pleasant + mainstream + cross-platform languages for new client/desktop software today are Go and Rust. Depending on the Python interpreter or the JRE or whatever is a huge pain when you don't control the environment. Just spitting out a native binary really helps. (I say this as someone who doesn't personally find Go pleasant, but I understand that lots of other people do, and I respect the tradeoffs it made.)

To be fair, the line right after that is "I know, I know, that's an insane thing to say."

Guess and check :)