Memory safety absolutists
itsallaboutthebit.com
itsallaboutthebit.com
Now as a software developer I feel like this is even more important because I use libraries maintained by thousands of other developers that might also use applications that have these exploits which get their systems compromised pushing malware to thousands of other developers which end up compromising even more libraries.
I believe that memory safety should be the standard for software that thousands if not millions rely on and that it shouldn't be some political issue of X is better, Y is that, Z is something else.
But then again, social engineering is the primary source of malware spread so I don't know.
However, I should probably pipe down, as I would not call myself either one.
Memory safety is just little thing that makes sure that when you write code at 4am that it will not leak memory via trivial mistakes such as forgetting to free something, freeing something twice or passing a freed pointer. I believe AI agents shine here the most because the they do not get tired and are getting pretty damn predictable.
Latest batch of LLM's Linux had 423 vulnerabilities. Out of which 10 were Rust*. Would you prefer more or less CVEs?
But it's like seat belt analogy. It's a helper not a panacea.
* Granted Rust isn't in the entire kernel yet. D
https://security.googleblog.com/2022/12/memory-safe-language...
Put another way: if you have to rely on programmer skill or attention to detail in order to guarantee something, that will always be a losing bet, on average. The existence of a tiny percentage of programmers that can clear that high bar does not make it a valid strategy.
Also in a program similar to qmail, memory safety alone is not sufficient, a lot of bugs in sendmail were related to complexity issues.
I wouldn't go that far, what matters is the finished whole. Memory safety of the finished program is a critical factor and using a memory safe language makes it easier to achieve that goal.
However simply using a memory safe language doesn't make you a "Software Engineer" any more than using a certified I-beam makes someone a "Civil Engineer". What matters is that the finished structure/program meets the explicit and implicit requirements of safety, functionality, durability, cost, etc.
Not to mention that complex reliable systems are usually engineered out of much less reliable components.
Presence of worse bugs won't make memory bugs disappear.
Is how this kind of arguments always get received by security folks.
(Actually, I can't remember a single incident where somebody I knew was directly affected by a memory safety issue)
That doesn't mean the issue is nonexistent. See this article by Microsoft that shows the percentages of fixed CVEs that are related to memory safety.[1]
[1]: https://www.microsoft.com/en-us/msrc/blog/2019/07/we-need-a-...
So I think looking at this with regard to what actually matters for users is completely misleading. Rust fans running around touting memory safety as the most critical thing that overrides all other considerations, but having several hundred of dependencies in statically compiled software and teaching users to do "curl | bash" is actually a disaster for security. I am not saying that memory safety is not a good thing, but this distortion of reality is harmful.
There's a reason they offer "download and run this .exe" on Windows and "download and run this .sh" on POSIX platforms.
...plus, as they mention in the tiny "other installation methods" link, none of the distro packages are official and people have a nasty habit of coming to you for support even if you didn't point them at the downstream-patched distro package they found.
Given how much distro packagers have tried to contort Rust to fit into their pre-existing `.so`-centric packaging system, I can see why Rust upstream would want to downplay that option to avoid headaches.
Let's compare what some other supplementary package managers tell people to do:
As for "and also not that Rust's design makes it more difficult to package", give https://blogs.gentoo.org/mgorny/2012/08/20/the-impact-of-cxx... a read.
TL;DR: It's not a Rust thing. C++ suffers from the same problem when you use templates in your APIs... it just emits a .so file with no contents if you expose only templated APIs. It's fundamentally a problem of "Unless you're willing to spend Swift-level engineering money and still accept compromises Rust and C++ can't afford, dynamic linking in the presence of monomorphic generics is a problem even research languages haven't solved".
...same as the warts in async Rust. What you're seeing is a cutting-edge language slamming up against the limitations of what cutting-edge programming language research has figured out.
At least, with Rust, you've still got the dependency tracking and automatic RUSTSEC advisory tooling you don't get with header-only C libraries.
Are you suggesting "modern C" is less prone to memory safety issues, and that if MS used "modern C" then that would meaningfully reduce the 70% of security vulnerabilities it claims are caused by memory safety violations?
Now, maybe I'm only running into these algorithms because these are the problems I'm running into, but in the Rust community I consistently got the message that linked lists were a legacy data structure. In some cases they are, but when they do apply they do so brilliantly.
I know working in Zig leaves plenty of room for memory unsafety. Perhaps that's a bad thing. But I'm implementing an interpreter, and these algorithms are essential for it to run fast. I really do need to be aware of where every allocation happens, and what instructions the computer is running. So for now I stick with Zig, even though I know I'm opening myself up to memory exploits.
Also, while such data structures/algorithms are rare, and more importantly, they can be written once and used many times, someone still needs to write them!
Here is some online content on this side of Rust:
- Learn Rust the Dangerous Way - https://cliffle.com/p/dangerust/
- Learn Rust With Entirely Too Many Linked Lists - https://rust-unofficial.github.io/too-many-lists/
- The Rustonomicon - https://doc.rust-lang.org/nomicon/
I don't mean this to be a gotcha, but I think it's important to say that we can have these in rust too! The only difference is that the first thing we have to talk about is you the user can go about using an intrusive collection correctly. I love the cordyceps crate for this. If you want to be able to use your type as a linked list node, you must implement a Linked trait, the documentation of which clearly explains how to use it safely: https://docs.rs/cordyceps/latest/cordyceps/trait.Linked.html
Does cordyceps have a derive macro? I can imagine that helps a lot with correct implemention, though when it comes to linked lists I can see people wanting to do it themselves.
That documentation appears to be out of date. The `addr_of_mut` macro was elevated to a first-class language feature, `&raw mut` (there's also a corresponding `&raw` operator). These two operators differ from the usual `&mut` and `&` in that they create raw pointers rather than references; prior to the introduction of this feature (or the aforementioned macros that served as precursors) to create a raw pointer you might need to have done a cast like `&foo as *const` in order to create a raw pointer by casting from a reference, but this could have safety consequences if the temporary reference was to an invalid object. Therefore `&raw` and `&raw mut` were introduced to create raw pointers directly without introducing an intermediate reference, which was arguably the most subtle footgun in unsafe Rust for a few years, and it's nice that it's now addressed.
Things are slowly getting better here! '&raw'/'&raw mut' reference operators were stabilized a couple years ago.
Another ergonomic improvement in the pipeline is a better way to access fields behind pointers, something like C's -> operator. This is taking some time to design because there are things other than raw pointers that would benefit from a generalized field projection mechanism, like Pin, NonNull, and (potentially user-defined) smart pointer types.
As of Rust 1.82.0 [0] you no longer need to!
> I'm sure there's a good reason, as Rust tends to think these problems through, but it's very unintuitive compared to returning `&mut`.
The addr_of_mut docs [1] give a pretty decent explanation of its reason for existence; in short, it lets you get a pointer to something without needing to create potentially-invalid intermediate references. It (and &raw) probably aren't going to be needed if all you need is a &mut, though.
> Does cordyceps have a derive macro?
Doesn't appear to from a quick glance, and given what's in the safety section of the docs [2] it'd probably need to be marked unsafe [3]. Unsafe attributes are new to Rust 2024, though, so that might be a bit new.
[0]: https://blog.rust-lang.org/2024/10/17/Rust-1.82.0/#native-sy...
[1]: https://doc.rust-lang.org/std/ptr/macro.addr_of_mut.html
[2]: https://docs.rs/cordyceps/latest/cordyceps/trait.Linked.html...
[3]: https://doc.rust-lang.org/edition-guide/rust-2024/unsafe-att...
> As of Rust 1.82.0 [0] you no longer need to!
That's true but misleading. I'm pretty sure the question was why a normal reference is bad. You don't need `addr_of_mut!()`, but you do need `&raw mut`.
Oh, completely missed that interpretation. I think in that case it comes down to references being incompatible with the desired semantics for an intrusive list - & doesn't work if you want to modify what you're pointing to, and &mut doesn't work since you may want multiple things to be able to point to and modify whatever you're pointing to.
Complicating the grammar for no good reason. This should have stayed a macro.
I feel like these kinds of things should be transformations that the compiler can use to turn Provably Safe code into performant code, i.e. you write the safe code and, once the borrow checker is happy, the compiler does its magic. I have no idea how to codify any of it into a compiler pass, or whether it's actually possible...
Though in terms of practicality, Graydon Hoare has mentioned in the past that he isn't a big fan of complex systems, and so it's better to have a simple type system that covers 95% of the cases, and an escape hatch when it doesn't. I think that's still a fair assessment, but I've fallen down the proof rabbit hole so I am curious to see how far it could be pushed.
And you can statically verify [4] your raw pointers, so that you only need to use the run-time-checked pointers for the more exotic lifetime relationships.
(That said, for the situations where Fil-C acceptably solves your problem, it's probably the more practical, complete and well-supported solution. And since not many seem to be explicitly mentioning it, the recent Fil-C demonstration of memory-safe linux userspace is rather impressive, right? I've heard that IT security is a $100B industry. I'm guessing an inappropriately low proportion of those resources are being invested in Fil-C. :)
[1] https://github.com/duneroadrunner/SaferCPlusPlus-AutoTransla...
[2] https://github.com/duneroadrunner/SaferCPlusPlus#norad-point...
[3] https://github.com/duneroadrunner/SaferCPlusPlus#tnoradproxy...
EDIT: though I'm considering switching to a tag + 64 bit union, because of 54 bit addressing and pointer tagging. It makes it tricky to get my data layout right though, since it makes all of the structures larger.
Apart from the utility, it is also a lot of fun. So it would be a pity if people are told that these things are legacy and they should not try to use/implement them..
This is a limitation of the rust model. You could have a language where intrusive linked lists are provably safe.
That's not to say rust doesn't work, but it's the first language I'm aware of that's attempted compile time memory safety, and no body seems to be talking about how to do it better.
(part of this I think is some C teaching which tended to overemphasize linked lists because they are useful for teaching the concept of pointers, which gave a lot of people learning C the impression that it should be the default way to represent a list of objects, when it's far more the exception than the rule)
Cache cost of individual allocations can be an issue but that can also be mitigated with a good allocator and even without special care can be quite performant in many applications.
I agree the linked lists shouldn't be the first structure you reach to. But there are situations where they can perform very well.
1. CS professors teach them. The Programme Lead for our CS course still teaches linked lists as the first data structure in the DSA course. Does this type exist? Sure. Is it a good idea? Almost never. But you wouldn't think so from its prominence in the course materials.
2. The Linux kernel uses a LOT of linked lists. Multi-core atomic operations on mutable data, a nightmare you will probably avoid in your software is necessary for the OS kernel and so it has linked lists. Linux is very famous, but your software is almost certainly not an operating system kernel.
In general yes, but right now I'm contributing to folk.computer, which is a multithreaded task scheduler/central DB (among other things not relevant to the topic at hand). The biggest use of atomics is during statement insertion and removal, by using RCU operations in the central trie.
One of our current performance drags is the interpreter we use is not threadsafe, so we have to serialize and deserialize objects as they move between threads. This contributes to about 30% of Folk's CPU usage. So I'm currently working on making a threadsafe interpreter by porting the Tcl interpreter we use (Jimtcl), and I'm learning all the fun things around atomic ordering and threaded data structures. So I'm definitely the audience for these data structures.
I don't know if Zig has made any clear decisions about this (chime in any Zig experts) but in C [and C++] the pointers are crap because they're "zapped" when the thing pointed to is gone. Now of course in reality your compiler won't overwrite your pointer just because the standard says it could, but because the compiler knows it would be allowed to do that, it can optimise pointer tricks in surprising ways. To work around this, C and C++ provide pointer-sized integers and exhort you to do any tricks with those instead because there's no zap.
In Rust that's unnecessary, the pointers are just pointers, if the thing pointed at is invalidated, you mustn't dereference that pointer because it's not pointing at anything now, but you can still for example XOR it against another pointer. Clever pointer tricks are thus your responsibility as programmer, and you can just do them with pointers rather than needing this pointer-sized-integer workaround.
Working with pointers in Zig is much nicer, since
1. They're not nullable. If you want a nullable pointer, you would use `?*Object` instead of `*Object`.
2. They distinguish between single item and multi-item pointers. `*u8` is a pointer to a single u8, so I couldn't add to it or index it. `[*]u8` is a multi-item u8, which you can index.
3. Pointers track their alignment. I could make a pointer with `*align(128) Object` if I wanted to tag the bottom bits for example. Zig will require casts when changing alignment, for example casting from `*Object`.
4. Everyone uses slices when possible, pretty much Rust's slices.
5. A smaller detail, but there are sentinel pointers/slices, where you guarantee that the last item is a certain things. For example, a C string would be `[*:0]u8`, because it ends with 0.
Zap isn't much related to provenance, it's more of a difference in philosophy about what pointers are for. With zap they're a very bare bones reference type, they always refer to things which exist and that's all, it's important that they never point anywhere without in fact referencing a thing. Without zap most of the behaviour you associate with hardware addresses works, you can do arithmetic on them, including atomic compare operations, you can use them as parameters to MMIO features and so on. Since C doesn't have any other reference types their role in C makes sense, in Rust and C++ of course actual reference types also exist.
If you want to go deeper, skip the language wars entirely. Tooling does what these languages can't do. I have had great success model checking C with CBMC. Not only does this prevent memory safety issues (including memory leaks), but this also prevents deadlocks. Bonus: you can write user contracts and invariants to verify that every execution path of a function fulfills these contracts and invariants.
Kani comes close with Rust. It doesn't yet have decent concurrency support, but there is nothing that prevents this from being added in the future. The ability to write custom contracts and enforce custom invariants more than makes up for its lack of concurrency support. Similar technology could be used or adapted for Zig or C++.
Use whichever language you like. Just, please look into model checking it. The technology scales just fine, once you get over the learning curve and learn how to use compositional verification.
I think the limits of Rust’s memory safety are interesting to discuss, as are the limits of Fil-C’s perf and practicality.
It’s best to discuss these things rationally, rather than accusing folks of trying to draw attention
> I think the limits of Rust’s memory safety are interesting to discuss, as are the limits of Fil-C’s perf and practicality.
Notably absent from what you claim to be willing to discuss are the limits of Fil-C's memory safety, or the possibility that it could be less safe than Rust.
What you're doing is well explained here:
When you allocate space with Fil-C's "malloc", some additional bounds checking data precedes the space the program gets to use. Pointers are "fat pointers", with a pointer to the beginning of the buffer and a pointer to someplace within the buffer, allowing ordinary C pointer manipulation.
There's much new verbiage around this. But it's roughly the same idea as GCC "fat pointers".[2][3] So it's not a new idea. It's one that's been tried several times, but never caught on.
There's a performance penalty. Especially if the compiler can't hoist the checks out of loops.
[1] https://fil-c.org/invisicaps
Fat pointers show up inline in memory, which has a bunch of problems:
- sizeof(void*) changes
- either you let the bounds get corrupted by bad casts, unions, and other issues, or you impose restrictions that prevent unions from working compatibly, or you end up supporting unions by having issues with races, or you need special hardware. Invisicaps sidestep all of those issues
The closest technique to invisicaps is softbound, but that has issues that invisicaps resolve (better story for races, more comprehensive safety for all of the C and C++ languages, no need for large virtual memory reservations, and lock freedom)
Coreutils has a good track record. Ffmpeg doesn't. They should have rewritten ffmpeg in Rust, not coreutils.
There are other approaches though like formal verification.
the reference C code is just as bad.
See Verve OS from Microsoft Research, TAL and the origins of the Dafny language.
Wait for an AI company to promote their new model by porting the entire ffmpeg to Rust (half ironically).
for foo in bartab {}
I used to argue with the C++ committee about to when it's OK to catch an error early. If you write
#define LEN 1000
int tab[LEN];
for (auto p = tab; p++; p <= LEN) { *p = 0; }
I argued that it's legit to catch an error at the point it becomes inevitable. This leads to bikeshedding objections: "But what if a signal interrupts the loop before it runs off the end". That's why you need a language where undefined behavior has been nailed down to do this optimization properly.
#define LEN 1000
int tab[LEN];
for (auto p = tab; p++; p <= tab+LEN) { if(*p == 42) foo(); *p = 0; }
The current standard allows the compiler to hoist unsafety only if there's no possibility of I/O or volatile memory access. This is the reason for the rule that infinite loops without the above are undefined behavior - it allows the compiler to merge two loops, the second of which might crash and the first of which can't be proven to terminate.
I don't like having to contort my code to fit each unit of work into the API of `std::panic::catch_unwind` so I can responsibily distrust the transitive dependencies beyond the reach of my Clippy lints.
Rust is pretty much state of the art in this area for general-purpose, non-GC programming languages, and people still complain about the effort involved in writing memory-safe code with it.
If you really want complete memory safety, use a garbage collected language.
Like, of course you can prevent all memory safety errors by using a garbage collector. That has been known since the 90s. In fact, there was a good two decades after Java released where all the research on memory safety just stopped, because the standard answer became "use a GC". If you couldn't use GC, you were stuck with languages made prior to Java - which in practice meant just C - or the one language everyone tried to staple every new programming paradigm onto, C++.
In fact, part of why C++ became such an untameable beast of a language is because it became load-bearing for non-GC projects. Anything not in the ISO C++ standard was, in practice, something you just couldn't do in native code. Oh, of course you can't introspect structs in a compiled language, of course macros are useless and unhygenic, of course templates have to be monomorphized and bloat your binary size. And so the pressure for C++ to be an all-singing, all-dancing, all-dressed language grew.
Rust's big story is memory safety, but the more Rust I wrote, the more I realized that the memory safety is only part of the picture. Rust has a very nicely curated selection of features that allows the language to remain understandable despite the sophistication of the compiler. Memory safety is a selling point, sure, but it is also the lubricant that makes working with those features pleasant.
If you want a real complaint about Rust, it's that some features are oversimplified in ways that make certain scenarios harder and make some features way more "magic" than they should be. Have you ever tried writing Futures code without making use of the async keyword? It's nearly impossible, for several reasons; the main being that Rust's type systems cannot express self-referential borrows. This also makes returning a reference to something in an Rc or RefCell you own more difficult[0]. And there are numerous other states memory can be in that are hidden from Rust's type system. Rust can't even represent a real destructor fn. Dropping a value multiple times, or using it after it's been dropped, is explicitly forbidden; but Drop impls still can't take values out of themselves because Rust doesn't have an "owned reference" - i.e. memory you can take values from but can't deallocate.
But none of this compromises memory safety - it just makes certain things harder than they should be.
[0] Strictly speaking, there's an owning_ref crate that manages this; stdlib is also working on a "mapped mutex guard" type that would do the same thing without a dependency.
I think the "static analysis" and the "runtime checks" approaches are complementary, not in opposition, making any noise around having to choose one or the other moot.
Same in regards to WG21, while more active in improving safety, there are plenty of politics between the profiles approach faction and everyone else.
No, it's been known since GC was invented.
> In fact, there was a good two decades after Java released where all the research on memory safety just stopped, because the standard answer became "use a GC".
Having done all of my research on memory safety after Java was released, I find this statement a little exaggerated... (e.g., https://barnowl.org/research/pubs/98-pldi-regions.pdf, https://barnowl.org/research/pubs/07-hotos-linux.pdf)
AT&T even had a full OS, where C only had minor role, the microkernel and Dis VM/JIT, with the whole userpace implemented in Limbo, designed by the creators of UNIX and C.
However none of them had an OS available in source code for the symbolic price to send tapes around to universities.
Additionally, the remark that the UNIX and C authors themselves did not stop there, and their latest mark on the computing world was the Go programming language, after Limbo, and the acknowledgement that designing Alef without a GC was a mistake.
Probably they would not had to reboot Plan 9 OS design into C.
As the parent comment indicates, it seems like there might have been more languages that were less known that did try to apply some of these ideas, so maybe I'm misremembering the history that I had heard, or maybe I did remember what I heard right, but the people who originally expressed it were similarly unaware that some of it had been tried already. Hearing more context about the time period I was referring to is definitely interesting, and I appreciate it being shared!
Lol, have you ever watch Andrew Kelley speak? I've only seen 3 or 4 talks he's given, but he frequently makes not-so-subtle digs towards Rust. Zig's home page prominently displays "Focus on debugging your application rather than debugging your programming language knowledge" which sounds horrible to me, but is clearly an anti-rust statement. Competition is good, and Rust and Zig cater to different people, but when the primary personality behind Zig is leads the way, it shouldn't be a surprise that some tribalism-influenced followers lean hard into it.
I responded to “clearly an anti-rust statement”, which is a bit of a stretch.
This sounds like it's made by someone who fears discovering they don't know something more than discovering they made a mistake.
I am the reverse: I would prefer to debug my programming language knowledge over debugging my application. I hate debugging applications specifically because making a mistake is SO much worse to me than discovering there is a fact I didn't know yet.
And, of course, debugging your application is discovering there are unknown unknowns in deployment, the worst of the quadrant of knowns and unknowns.
Garbage collection and a higher-level language that controls allocations handles 2/3 of the memory-safety problem space (using memory you shouldn't via use-after-free, or using memory you shouldn't before allocation/via arbitrary address access), but the other 1/3 isn't addressed by GC: out-of-bounds access on properly-allocated structures.
GC-or-not doesn't stop a pointerful language from addressing memory off the end of an array, and the best techniques we have to deal with that today are imperfect (but still pretty good!) compiler inference for BCE, memory protection to limit the scope of possible overreach, or slowing down runtime by inserting checks.
The rest of your points are well-taken; just pointing out that GC isn't the solution to all causes of memory-unsafety, just 2/3 of 'em.
...of course, the managed code languages had an answer to this: put a lock on every managed object you allocate.
But they didn't actually make you acquire the lock - it only comes into play if you declare a method to be synchronized or if you acquire the lock externally. Which is a problem because most of these languages ALSO are designed to act as a security boundary for untrusted code - something Rust doesn't even offer! Which means even though the language doesn't enforce race-freedom, it has to enforce enough of it anyway to keep racy code from corrupting the language heap (which is now security-sensitive). No other language (not even Rust) does this - if you write a race in unsafe Rust, it is actually UB, but in Java you actually still have some guarantees about what your program will do.
Actually, let me throw a bone to the Zig people: Unsafe Rust actually imposes more UB than C does, and it is more difficult to write sound unsafe Rust. If you're writing FFI code[0], you're probably fine. But if you're writing new memory abstractions, you're probably going to run into Rust's requirement that all mutable references be exclusive. If this ever fails, your program is already in UB and all bets are off. Morally speaking, Rust sprinkles the equivalent of C's restrict keyword over every &mut in your program. In fact, this UB is so strict that the Rustc people have had to turn this on and off multiple times in the past because LLVM would miscompile sound / safe Rust code if it was told about the aliasing restriction.
In practice, however, this doesn't really matter. Rust has all sorts of soundness holes nobody would ever trigger by accident. There's a long standing trait-handling bug that lets you confuse memory types in safe Rust; and any OS that exposes process memory as a writable file lets you do the same thing. In Java, at least the former would be a CVSS 10.0 security vulnerability, but it doesn't matter in Rust, because safe Rust is not a security boundary. It's a set of tools to keep you from shooting yourself in the foot. And, in practice, Rust does a pretty good job of that.
[0] Or more generally, unsafe code where you have two unsafe pieces that you have to guarantee are used together. For example, if you have a C-style callback API that takes a function pointer and a data pointer, and then calls the function with the data, you have to use Box::into_raw() and Box::from_raw() to maintain ownership over the data. from_raw is unsafe, but all the obvious ways of using it are sound.
All relevant scenarios when the goal is systems programming.
Rust does not prevent race conditions. You're getting confused with data races.
However GC's solution to data races (make every load/store act as very relaxed atomic instructions) makes race conditions much easier to write.
I'll take the bait then. Here's rust official documentation stating that it doesn't prevent race conditions: https://doc.rust-lang.org/nomicon/races.html
Since the 60s actually, starting with Lisp, Scheme, CLU, Cedar, Smalltalk, Cedar,...
owning_ref is unsound and shouldn't be used. self_cell/yoke/ouroboros are much better alternatives, ableit more complex
Honestly, it gets tiresome and adds nothing useful to all these posts. And since those drive-by missionaries aren’t spreading the good word of ocaml/java/go/haskell but Rust - well, then eventually you tire of the Rust community. It has a missionary/redemption problem.
Combined with the ability to compile to a single native binary, having a solid package management solution, a fairly advanced type system, and a solid selection of nice language features. Usually you need to give up at least one of those.
Good to see a praise for C#.
It’s tiresome when other language communities start engaging in a battle with a vocal minority of another community.
Every language has its annoying maximalist pushers. The play is to ignore them and do what decisions are best for the language, not to let yourself get dragged down into petty language wars where nobody wins.
I'm hoping that the next release of Sculpt, Genode's user-facing OS release, will offer this, as promised in their road map for their realse 26.08.[1] I'm hopeful that we can finally drop all these stupid layers of cruft trying to patch fundamentally insecure operating systems.
Tannenbaum was right, in the end, and Linus was wrong.[2]
[1] https://genode.org/about/road-map
[2] https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_deb...
[*] "Multi-level secure operating system" is the required magic phrase to allow finding articles on this subject through search engines.
Not so much about language design as how easy it is to take a defensive position on something we are comfortable with when it's challanged. This prohibits growth.
When someone suggests you have been doing something the wrong way or has a new idea, it's usually better to hear them out and take what knowledge you can from them, even if they weren't actually trying to help you.
Anyways, thanks again
[1]: https://docs.scala-lang.org/scala3/reference/experimental/cc...
Yes there is Cranelift, and?
Also Fil-C can't be used for LLVM anyway, since performance and memory consumption overhead is way too much. Nobody wants clang/rustc working 4x slower and consuming 2x more memory.
GCC, GNOME, KDE, Linux kernel, BSD variants, CUDA, RocM, OpenCL Vulkan, DirectX, Metal, GNM(X), OpenMP, OpenACC, NVN,....
Hence why somehow fixing C and C++ is also quite relevant for the upcoming decades, assuming humans still matter on the planet.
Now it is going to be ARM MTE, SPARC ADI, CHERI, Fil-C, or WG14 and WG21 actually getting their act together, remains to be seen, as it is subject to governments and industry pressure, and we not destroying civilisation.
Isn't needed if we rewriting anything in Rust anyway.
> GNOME, KDE
They aren't that huge. Huge is the overall codebase of applications using them. It's relatively easy to create a native Rust GUI framework and rewrite applications needed such a framework one by one.
> Linux kernel
It's a good opportunity to rewrite it. Not only because modern languages like Rust are better than C, but because such a rewrite allows getting rid of old stuff present only for historical reasons.
> BSD variants
Aren't needed if we writing a new kernel from scratch.
> CUDA, OpenCL
Not a huge problem. There are dozens of GPU-related languages. Adding one more isn't that problematic.
> Vulkan, DirectX, Metal
They are language-agnostic (but still unsafe). One can create an implementation in something other than C.
Likewise with all the industry standards based on C and C++.
Looking forward to see how COSMIC ever manages to gain adoption beyond System 76 computers.
Fil-C is an amazing technology, and we should use it, but we should also recognize its limitations.
An as-written JIT or GC can not be compiled with Fil-C because they are inherently memory unsafe.. which means one of the biggest user CVE targets by volune, Google Chrome, not only can not be compiled by Fil-C -- it also can't consume Fil-C compiled libraries. (Chrome not only has v8 jit, but 2 different GC systems - V8's and Oilpan, plus PartitionAlloc)
Which means Fil-C is an amazing tech for securing the manual-deallocator daemons and backends, but does not secure the biggest attack vector on 2 billion consumer devices.
This is why i'm writing a maximally memory safe browser in dotnet where the only unsafe will be the FFI and RyuJIT underneath me. [1]
I also think we have gone too long with poor operating system safety tools. I want a system based on Andrew Valencia's VSTa's hierarchial capability system. In that system protection ids are hierarchial and infinitelly narrowable without any special authority. So user.david can forge user.david.browser which can forge user.david.browser.site.ycombinator and so on. The closest we have to this today is (relatively) heavy weight virutlized containers.
I realized some time ago when I talk to people about rust, I don't really mention memory / thread safety in a security context, I mention them in a reliability context. Compile-time checks mean I basically never spend time debugging runtime crashes, and generally can count on the compiler to catch memory and thread safety issues before the program ever runs, and this saves me time and aggravation in production. I care about security, but I care more about reliability, and rust compile-time checks mean my software is reliable in a way that runtime-checked languages just aren't.
This applies to lots of languages. Every time I hit a runtime crash in python, or ruby, or js I die a little inside. Even though they are memory-safe because they're garbage collected, I still waste time debugging runtime errors.
Memory safety is a really big reason to use Rust, but it is very very far from the only reason. Even if Fil-C was magically zero-overhead (that is basically what CHERI is), I would still rather use Rust.
Rust has so many advantages over something like Fil-C it's hard to list them. I think Fil-C is a great project but it's only really relevant if you have no choice but to use C. If you can use Rust you also get:
* Compile time memory safety (Fil-C / CHERI are run-time).
* A modern functional-style type system (helps prevent logic bugs).
* Tree ownership (helps prevent logic bugs)
* Sane toolchain (helps prevent hair loss).
* Easy dependencies.
* Vastly more things checked at compile time.
Also I find it amusing how this post claimed it wasn't about Rust and then spent the whole time talking about it.
Rust is great. I don't think that really needs to be debated any more.
That's true, though it's worth noting that they do still need unsafe for the FFI. Often this is where the memory safety issues arise in GC languages. So no language is truly "memory safe".
Execution is only allowed after the OS admin adds the executable to a specific process white list.
This is similar to how some managed runtimes work, and the Java ecosystem is moving forwards it.
Currently it only triggers a warning, however in the future JNI and Panama will be disabled by default, and like with ClearPath MCP, it is up to the person installing the application to enable the unsafe layer explicitly.
Another example is the capabilities approach on WASM runtimes, here again the one executing the platform takes ownership on enabling unsafe behaviours.
The customers that still buy such systems, care about security as the feature above anything else.
As you can see, the marketing is all about security.
https://www.unisys.com/product-info-sheet/ecs/clearpath-mast...
To the extent Fil-C and Rust both address these vulnerabilities, and don't include design features that in any practical way admit them, they're memory safe. What always feels like is missing from these kinds of analyses is that there are lots of memory-safe programming environments. Almost every Java, Python, Ruby, Javascript, and Go program is memory safe, in the real meaning of the term.
The competition to lock in and promote adoption of the "most" memory-safe language is a category error... unless you do what every language-war argument does, and redefine the term.
I never seen this happen with GC/RC languages that also support unsafe code blocks.
Unfortunately, I've never seen a version of this data targeting modern C++ (>=11, with smart pointers, already 15 years old).
Most projects keep being full of C idioms, regardless of how much advocacy we keep pushing for.
There's no True Scotsman C++. All C++ codebases would be perfectly safe, except the ones that people actually wrote.
Note that Rust has been created after C++ already had smart pointers. Rust treated "modern" C++ as safety failure to be replaced, not as an unrealized competitor. This is still a problem with the C++ WG today: their most ambitious safety ceiling is aiming below Rust's floor.
You could even use Rust alongside Fil-C to ensure the unsafe blocks are safe. It would even be interesting to turn off some of Fil-C's expensive protections in the safe parts of the Rust code, making an average-of-both-worlds sort of solution.
What I do care about are C and C++, these two absolute garbage languages (though I do have a lot of love for C, you can both love and hate something).
Tools like Fil-C, hardened mallocs like SlimGuard, and other "make C safe" solutions all sacrifice absurd amounts of performance because these two moronic languages refuse to have a proper slice/span type.
Use-after-free and double-free are serious problems but they're a spec of dust compared to missing bounds-checks. The low-hanging fruit is right there for the picking but everyone is worried about how to reach the fruits at the top.
C++ only now in C++26 is finally adding bounds checks to the [] operator on std::span and (hilariously) is also finally adding the now mostly redundant .at() method which should have been there from day one.
Clang added -fbounds-safety which is a feature that should have existed for a long time and serves as a decent stop-gap to a proper slice type. But of course, since it's not standardized, most people won't use it.
What these two languages have taught us is that if you want to produce utter garbage you should make it an ISO standard.
Me, personally, I find this whole discussion on memory safety amusing. Missing bounds checks are by far the biggest source of vulnerabilities and they're a problem in exactly 2 languages and those 2 languages have outright refused to do anything about it for decades.
But hey, even in memory safe languages you have frameworks like log4j that had remote code execution from a format string as a feature. Is the problem memory safety specifically, or this utter cavalier attitude towards security?
I'm not even suggesting something dumb like "you just gotta get good at C and then you won't have any vulnerabilities". Humans are fallible, which is why we delegate what we can to machines. I'm just pointing out the utter lack of care. People just don't care.
At least do the bare minimum of effort like, I don't know, not make remote code execution a feature tied to format strings. Or provide a slice type in your language and string manipulation functions in your standard library that make use of it, preferably 20 years ago.
Isn't this what MIRI already does?
Miri is meant as more of a sanitizer type tool that you use during development to catch bugs. Fil-C is a platform you compile your entire Linux distro with for use in production.
I don't see a point in running Rust code in Fil-C. Due to the way Fil-C works it won't be enough to check the `unsafe` statements, so you'll pay the performance tax on safe code too (onto which you already paid the borrow checker tax!)
It is also relatively strange that many devs use the lacks of standardisation as an excuse to not adopt safety features in C and C++, while at the same time rush out to adopt cool toys that are specific to a single compiler.
Here is for VC++
https://github.com/microsoft/STL/blob/2a62bf7b4079f0a3e33ec8...
You just have to define the proper iterator level for enabling bounds checking.
Other compilers have similar approaches.
I chose VC++ on purpose, because it is what I know better, and is famously trailing behind C++26 features.
Then there are the Tclass, Cclass, Qclass, from all those C++ frameworks pre C++98.
However plenty devs seem allergic to learn about how to use their compilers.
I don’t dislike Rust, and when I point out that Rust is not fully memory safe, it’s because I find the details here super interesting. It’s interesting that Rust deliberately chooses to have an unsafe subset. It’s interesting how that leaks out to the rest of the language. It’s interesting because us language designers ought to be thinking about how this could be avoided. It’s a hard problem! And that makes it fun!
If you do read what I say on twitter, you’ll find praise for Rust, many concessions about Rust being faster than Fil-C, as well as a wide range of other opinions. When I point out Rust’s unsafety, I’m just citing facts. It’s interesting how doing that really seems to upset some folks.
Pointing out a limitation in a technology does not imply dislike, and it’s best not to take it personally.
As someone who really enjoys seeing all of the different programming languages and their approaches to different problems, it’s becoming obnoxious to hear the constant battles among people who think programming languages need to become part of your identity. The endless battles of superiority and tit-for-tat responses feel petty and distracting.
It’s refreshing to get back to people who just want to try different things and experiment without making everything into a battle where each side scores points against the other.
We should do more of that as a community. It’s important stuff. Ima do my part so you’ll likely see me poke at how Fil-C does a thing that Rust doesn’t do. If you read my arguments unemotionally, I think you’ll get a deeper appreciation for Fil-C, Rust, and memory safe language design generally.
What we shouldn’t do is reduce the discussion to claiming in a blog post that so-and-so “doesn’t like” such-and-such.
FWIW, I can see where TFA is coming from and I had the urge to write something "in defense of Rust" after listening to / reading some of your recent comments. I think your sister comment about the limits to Rust memory safety being real as much as the limits to Fil-C's performance and practicality is spot on, but the feeling (those emotions again) one gets from much of your commentary is that you downplay the latter quite a bit.
I think the real argument for Fil-C is that you aren't going to "rewrite in Rust" but you can often realistically recompile in Fil-C and pay the performance penalty; the fact that that Rust rewrite could also end up less safe is I think should be argued mainly from the AI angle where the only practical way is an LLM based rewrite but that would have a load of unsafe - way more than a "proper" rewrite - to be practical and the recent $100K Bun rewrite is a testament to that.
(I would love nothing more than for Fil-C to become as widely used as it practically can, since no other approach does nearly as much to secure C / C++ code and a lot of said code is out there; and security aside, it could prevent a lot of data corruption that most major C++ programs gift to their users.)
Not only languages, editors, OS, hardware platform,...
Because when it comes to apply for a job, the HR drones don't understand that one can manage to program in more than one language, use more than a specific OS, and so forth.
It was to be those specific bullet points, and by the way at least during the last five years exactly before applying to the position.
There are also the trivia questions during the interview about obscure language features.
So this naturally spills into one's identity, going back to the primitive mind of the days humans lived in small groups and it was us versus them across all tribes.
Which for those of us that are generalists, is a big pain in some body part.
I do. Not for any technical reason, but because the Rust evangelists seem ready to burn me at the stake if i don't agree that memory safety is the one true god and Rust is it's prophet.
They hurt the language adoption more than they help it.
The problem I have with the framing I've seen countless times from you on social media is that I have never seen any nuance on what Rust's memory safety model achieves. You say you state facts, but stating facts without context can still be dishonest. Claims like "both C and Rust are memory unsafe languages" or "both C and Rust allow introducing memory safety vulnerabilities" are factually correct, but pretty much useless in practice.
I'm sorry for misinterpreting the "dislike" part. In retrospect I shouldn't have even brought like or dislike into it cause it was not the main point anyway. The main point was framing that I've seen on Twitter, not only from you, that in my opinion is quite one-sided.
I removed the dislike part from the post now.
I have added the nuance. Folks now understand that there are limits to Rust’s memory safety.
What have you added to the conversation? Nothing.
This doesn't strike me as particularly constructive discourse
One of the top two issues with Rust is its cumbersome and overbearing syntax and anything which sidesteps that is automatically attractive to anyone which is worried about memory safety and doesn’t like to encode every little detail about in the type system. That’s the majority of programmers… think about it, that’s one of the big reasons people switched to GC languages which are the most popular languages.
The classic Rust approach to addressing criticism of the syntax was a mix of downplaying, gaslighting and “you’re holding it wrong”. But if easy to use, reasonably ergonomic alternatives become available, I expect that most would prefer them to Rust.
My system sometimes detects a few bitflips per day.
Perhaps you should replace your RAM, or it's a sign that you have a massive source of radiation lurking somewhere?
It wouldn't be the first time engineers pushed something close to the unreliability limit and compensated with a mechanism to make it more reliable.
I also feel there's a second dimension to the politics, where Rust is the "woke" language and C / Zig / Odin are now the "anti-woke" languages. At least, going by their most vocal online communities.
I'm sure offline there's still engineering decisions being made (I hope).
The most toxic Rust leaders exited from the project a few years back, though some are flaming more freely than ever, e.g., Go barely deserves to exist, “SQlite is a terrible example” etc
(And it's entirely possible it was back when I ONLY saved a copy of articles of that tier of "I may need this again" via ScrapBook/WebScrapBook, I haven't yet set up the browser/indexer for those and a few quick ripgrep searches aren't turning it up.)
I believe it had something to do with the JVM having some flawed semantics which couldn't be fixed without breaking compatibility with existing bytecode.
I’ve only heard the woke/anti woke framing from Lunduke on the right, and a few Rust fanatics on the left. Both parties and their sympathizers want to fit the Holy Language war into the Culture war™
As an example of that, the rust compiler helps the programmer to avoid many mistakes but it's still possible to use memory incorrectly and the tool is only safe as long as you use it as intended. You could say that C is safe too as long as you use it as intended and make no mistakes. The only problem is that it's hard to use it as intended and make no mistakes.
So let's think about what memory safety could mean. My understanding is that a memory bug is when you use memory in an unintended way. An example of that is when you write to memory after you have deallocated it, which means after you have decided not to use it any more.
No compiler or tool can make sure you don't write to or read deallocated memory, because whether it's allocated depends on what your intention is, and no compiler can read your thoughts. They can only give you a tool (like functions for allocating and deallocating memory or compiler checks) and hope that you will use that tool in a way that reflects your intention. One problem is that those tools are not always sufficient to keep track of your intentions and you may not always use them as intended.
Memory allocation is relative. In a sense, no program that runs under an operating system can use deallocated memory, because when they read or write to memory that has not been mapped to the process, they crash. In a sense, even "safe" rust can use deallocated memory; let's say you have an array of 10 integers and you decide that the fifth integer should not currently be used, you have then deallocated it on a level where the available tools can not help you, but you can of course still access that memory.
So again, everything is safe if you use it as intended and nothing is safe if you use it in other ways. Safety depends on how well the code matches your intentions. Remember that programming always happens in layers and no tool can cover all of them. It's not even clear what "all layers" would mean or how many there are in a program.
Only by misusing unsafe. Using it in regular programs actually isn't that necessary. In languages like C you have unsafe code almost in every line.
> My understanding is that a memory bug is when you use memory in an unintended way
Memory safety rules are more strict and formalized than you think. Memory safety issues are typically reading uninitialized memory (or strictly speaking changing observable behavior based on contents of uninitialized memory), reading/writing memory after it have been freed (free/delete call or out-of-scope going for local variables), concurrent unsynchronized memory access.
> No compiler or tool can make sure you don't write to or read deallocated memory
I am author of a programming language, where you can't write to or read deallocated memory, unless you misusing unsafe.
> no compiler can read your thoughts
But many languages more complex than C have powerful features allowing telling the compiler your about intents. At least partially.
> Memory allocation is relative. In a sense, no program that runs under an operating system can use deallocated memory
You are mixing two different concepts. One is memory model of the abstract machine defined by the specification of a language and other is the OS processes model. They have much in common, but there are a lot of differences.
> So again, everything is safe if you use it as intended
Such mindset is considered harmful, since it provides an excuse to use languages where making mistakes is easy (like C).
So which layer should we care about? All of them? Yes, if we want the most safety.
I will give you an example of how I can use memory incorrectly in "safe" rust. Let's say I have an integer that contains my age. Now I forget what the number was used for and I use it as shoe size. I have now used memory (the content of the variable) as something it was not intended for. Maybe you think this example is silly, but what is really the difference between using a float pointer as an int pointer and using age as shoe size? Another thing we could do is to use an index for one array as an index for another array. That's an example of using a pointer as the wrong type because an index is just a relative pointer.
So if we want to care about all layers of memory safety then we have to care about not using ages as shoe sizes and indexes with the wrong arrays. "Memory safe" languages usually don't help us with that and therefore they don't give us total memory safety. I think "memory safety" is not a very useful term because all data in a program lies in memory and basically all bugs have something to do with using that memory in wrong ways.
In languages even slightly better than C one can create a wrapper type for int/float with additional semantics like age, shoe size or something else. Since they are different types, using one in place of other isn't possible. This doesn't solve all problems, but at least can prevent silly mistakes.
`type Shoe_Size is new Integer` defines a type incompatible with `type Age is new Integer`
(The first ShoeSize is the type name, the second is the name of the constructor that converts an integer to one, or pattern-matches on it. This is unambiguous in Haskell and they are often the same. It has symmetry with "data" which is for more general ADTs that can have more than one constructor.)
On an academic level, memory safety is split into "spatial" and "temporal" cases and one that Wikipedia calls "spatiotemporal", with reading off the end of "capacity" being a spatial memory safety bug and reading within "capacity" before initializing being a temporal memory safety bug.
The other temporal memory safety types as Wikipedia enumerates them are:
* Use after free
* Double free
* Mismatched free (stuff like not using the free() from the same DLL that called malloc(), mixing up malloc/free with new/delete, etc.)
Or as the poster put it
> When seeing the title of this post, I bet in some people's minds, the first thought was "Rust devs!". This connection is not unfounded.
Exactly. The Rust community has been very holier-than-thou on the memory safety front and quite absolutist ("how dare you code in non-Rust, don't you care about memory safety? You must be a bad person").
Now that it turns out that there is an alternative that is even safer, we suddenly get "well, it's a bit more complicated, memory safety isn't everything, you have to look at the broader context and requirements"
Excellent!
Glad you've come around to that point of view, dear Rust community. Now let's have civilized discussions about trade-offs.
Well I feel most of the Rust devs are there for saying shits about other languages to prove their own dumbassness.
The real issue is that we came to accept unsafe languages and are taking a really long time to put such features back.
The problem with choosing Rust over a GC language like the above (or a good one, like OCaml) isn't that it's not "fine" to use Rust, but that manual memory management is an inefficient use of developer time. That's an issue for the developer, and one he inflicts on himself, not an issue for the end-user. Both Rust and (safe) GC languages provide memory safety, after all.
Really anything that deals with untrusted input should be memory safe. Your TLS library. A load balancer. Your password manager.
https://github.com/google/wuffs
WUFFS gives up generality - you can't write "Hello World" in WUFFS because it lacks both strings (for the "Hello, world" text) and I/O (for the printing it out). But you can write a codec, going from a block of bytes representing the encoded file to a block of bytes representing pixels, or PCM audio, or indeed uncompressed data [or vice versa] is easy.
But unlike Rust, all of the safety in WUFFS was checked during compilation. For example Rust emits bounds checks, arr[n] might panic at runtime if n is outside the bounds of arr, but WUFFS doesn't do that, it'll have proved mathematically that n is always in-bounds, if it can't prove that it rejects your code, make sure n is in bounds and try again.
Sometimes the end result is similar, you write code to check n at runtime, if it's a miss you report an error, WUFFS can see you met the criterion - basically the same as Rust. But often in a codec design you can just prove it's in bounds, if you implemented it correctly.
So... Yes, Rust is less safe. Just that now the Rust apologist wants to back away from that and say that actually memory safety isn't actually the end all be all. C has a lot of security problems. Rust has less, at the cost of breaking the ecosystem. Fil-C has even less, and breaks the ecosystem. Why is the place Rust stops now suddenly good enough?
TL;DR: Look at it from our perspective. It's not reasonable for us to be beholden to the limitations of build targets we never consented to supporting and possibly didn't even know about.
To apply your logically consistently, you'd also have to throw out Go[1], Java[2], C#[3], and plenty of other languages. I guess you can define a logically consistent taxonomy of languages where Fil-C is the only "safe" language any everything else is unsafe, but then the burden is on you to prove that it's a useful way of thinking about them. Meanwhile, the way that people who consider Go and Java and Rust to be safe and C/C++ to be unsafe will continue to see actual real-world differences in outcomes when using a safe language compared to unsafe ones.
As an aside, it continues to be absolutely wild to me to see how most arguments in favor of C/C++ over Rust nowadays are based on framing things theoretically rather than practical ones. It was not all that long ago that the situation was reversed, with people claiming that Rust's benefits were all theoretical when most C/C++ code would have all of the UB found and removed over time. Now that Rust actually has been getting used for real-world stuff for a few years, and the number of vulnerabilities found in C/C++ code does not seem to be going down any time soon, any actual empirical evidence gets handwaved away. Maybe Fil-C will be able to provide as large benefits to running C safely in production like its proponents claim in the future, and if so, that will be a solid argument against the utility of Rust in a lot of situations, but if we're collectively going to shift the standard to discount pragmatism over theory in this debate, we might as well not try to hide it.
[1]: https://pkg.go.dev/unsafe
[2]: https://docs.oracle.com/cd/E92951_01/coherence/java-referenc...
[3]: https://learn.microsoft.com/en-us/dotnet/api/system.runtime....
Yes? Is Fil-C not obviously safer than those?
> I guess you can define a logically consistent taxonomy of languages where Fil-C is the only "safe" language any everything else is unsafe, but then the burden is on you to prove that it's a useful way of thinking about them. Meanwhile, the way that people who consider Go and Java and Rust to be safe and C/C++ to be unsafe will continue to see actual real-world differences in outcomes when using a safe language compared to unsafe ones.
It seems like "can switch on unsafe whenever" vs "has no escape hatches" is an obvious line in the sand and a very useful distinction with real world implications.
> Maybe Fil-C will be able to provide as large benefits to running C safely in production like its proponents claim in the future
Why in the future? It exists and can be used right now. In fact, much of its appeal is being able to use existing code without needing to port to a new language; pragmatism seems like an argument in its favor.
Sure, although to me, the obvious distinction is that one of them gives you more flexibility (since neither Go nor Fil-C is at risk for someone accidentally writing unsafe code).
> Why in the future? It exists and can be used right now. In fact, much of its appeal is being able to use existing code without needing to port to a new language; pragmatism seems like an argument in its favor.
Because "can be used" is not a statement of present use, but potential future use. You seem to be missing my entire point about theory versus practice; plenty of things might be useful in theory but don't ever get widely used in practice (e.g. because they're not user friendly enough, or they require constraints that aren't acceptable to most users). I'm not making a claim about whether Fil-C has these issues, but making a claim that as yet, there does not seem to be widespread usage of it. If it's really so seamless to drop-in as a replacement for C that's strictly safer without downsides that make it unappealing to most people, I'd expect that to change. If that doesn't happen, I'd consider that evidence that it's not actually providing meaningful safety in the real world (in the https://xkcd.com/1312/ sense)
Just look at how long it took LLVM to catch up to GCC. Writing a competitive optimizer pipeline is hard work.
"No room for a language above assembly and below Rust" is a design goal for what it's reasonable to build with it.
If anything, using LLVM for the default backend is completely on-brand with how much priority Rust puts on being suitable for partially or incrementally rewriting C codebases.
(Sorry for the slow reply. The tab got buried.)