11 year old unsoundness bugs found in Rust borrow checker
github.com
github.com
A valid program would be exploiting undefined behavior in rust, but rust has no undefined behavior. Thus such a program must not exist.
Now a program may appear to work before the fix, but it was never a valid Rust program.
By Rice's theorem you cannot be sound and complete, i.e. accept all valid programs and reject all invalid ones. Most languages choose to be sound but not complete, i.e. reject all invalid programs but also some valid ones (C and C++ are the notable exception, although most languages have escape hatches). GP was saying that the fix might reject valid programs that compile today. This is true for some existing soundness bugs, but not necessarily for this one specifically.
That said, I was sarcastic.
You’re ignoring the possibility that Rust semantics are sound, and the implementation doesn’t fully implement them. In that case, we do not have to find sound semantics.
That, I think, by far is the most likely option.
Reason: I’ve not seen a single comment claiming the example should compile, and zillions (implicitly) acknowledging this is not what Rust should do.
> but not necessarily for this one specifically
> Maybe, maybe not.
But "the Rust semantics" are not defined, formally. The closest we have is what Rust implements currently, and that was shown here to not be sound. Other than that, we maybe have vague ideas in the brains of some T-types members, which are not detailed enough to determine soundness (source: those type system people want to formalize the semantics so they will be).
And for the record, we know of other soundness issues where the likely resolution path will involve rejecting valid programs we accept now, and the only question is how many.
Rice's theorem implies that it's impossible to precisely distinguish valid and invalid programs just by looking at their source code. Thus Rust rejects some valid programs in order to be sound (e.g. "if false { None.unwrap_unchecked() }").
In the case of the bug report in OP, I don't think that there are many valid programs in practice that exploit this soundness issue, so fixing it just means rejecting more invalid programs.
I think it's a good thing to keep in mind, that the actual implementation of Rust almost certainly isn't sound. Even an "ideal Rust" still has unsafe, which can affect the soundness of even faraway safe code. But I don't think it's nessesarily wrong to talk about Rust as if it were safe. Much of our communication has implicit assumptions, and the way we talk about Rust is no different. It would be exhausting to qualify every discussion of saftey with "assuming the compiler has no bugs", "assuming all unsafe code follows the (not fully specified!) rules", etc.
Ultimately it's like any other implementation of theory into practice. The theoretical model of Rust lets us reason about and design an implementation to achieve certain practical goals even though the model is a simplification of reality. In Rust's case, practical use has demonstrated it achieves design goals like reducing bugs and reducing development time (compared to peers like C++) to a substantial degree.
Python is powerful... and fast; plays well with others; runs everywhere; is friendly & easy to learn; is Open.Yes, you are all correct. The compiler may have a bug. And if it does not, then the type system may have a problem. And if it is formally verified, then the theorem prover might have issues. And if it does not, then the very math it relies on may be wrong.
But what do we contrast this to, what is the alternative to correctness through the type system? Almost every time, when it isn't "nothing", it's "tests". We rely on tests to test whether the program behaves correctly.
And as we all know, quite in contrast to type systems and their checkers, tests are always error-free, complete, and sensible?
There was one in the Linux kernel a while ago that mistakenly eliminated an important null check.
Is this different?
Here's the current list of similar (unsoundness-causing) open compiler bugs in Rust:
https://github.com/rust-lang/rust/issues?q=state%3Aopen%20la...
Now, you have a language sold as safe, with popularity made manily blaming C... but now we can say it gave the false sense of safety, distracting programmers from real issues in the name of its error messages that lead for endless refactoring if some new feature is added to project just to o have peace with the compiler. It is a language that program the programmer, an inversion of roles.
Except we can't say that, at all. This is just a single bug in relatively obscure code, so 99.99999% of your code remains safe.
Meanwhile in C, 100% of your code has the potential to trigger a similar bug, except there's no path for the underlying cause to ever be fixed, unlike here.
You're essentially complaining that something isn't 100% safe, but that's just your misconception of what "safe" means. Nothing is 100% safe, not can it ever be.
It's especially ridiculous to then compare something 99.999% safe to something 0% safe and conclude that all that "supposed safety" is useless, because look: in 0.001% of cases it's NOT safe!
Considering this has gone unflagged for 11 years... Finding a flaw in Rust’s safety guarantees doesn’t mean that they provide no benefit, over a programming language without them.
Ok, but that perspective is only valid from Rust++ users who get to mog both Rust and C users. It's not a valid perspective from C users.
Also, your position against Rust is really strange to me, because I love the lack of mutable aliasing. If you think it is a punishment for the programmer you're stuck in a programming model from the 70s. Rust's model is theoretically optimal both from a hardware design perspective and a compiler perspective.
VLIW compilers love Rust because the lack of mutable aliasing by default allows aggressive instruction reordering and instruction level parallelism! Software managed caches are only possible by essentially dragging the cache coherency protocol into the software level!
The sad news is that LLVM is a classic compiler backend optimized for mutable aliasing, which means it is not that great at producing code for VLIW architectures.
Note that Rust is not a theorem prover and Rust programs are usually not formally verified (there are some extensions like Creusot to verify Rust programs, but they are incomplete and a work in progress)
Compiler bugs are, like, just logic bugs. There's a project to eliminate compiler bugs that cause unsoundness, project zero [0] but, still, without formally verifying the implementation, there is nothing really guaranteeing there's no such bugs left.
(note: unsoundness in this context means having safe code cause UB due to a compiler bug; the fix is usually to make the compiler reject the code with a compiler error)
To make matters worse, Rust still doesn't have a spec, and there is not even a tight definition of what are the rules of unsafe code - that is, what rules you must follow so that you unsafe code doesn't trigger UB. Discussions around this are done mainly in issues of the unsafe code guidelines repo [1]
[0] https://goals.rust-lang.org/2026/roadmap-project-zero.html
Not that that would make any difference, theorem provers can also have bugs. Famously this summer someone “disproved” the collatz conjecture by exploiting a soundness bug in Lean.
The eventual formalization of Rust and a creation of a formally proven compiler is presumably going to be the end point of this process.
https://www.lesswrong.com/posts/zHhGa3iQBQcdSifFL/a-critical...
This 2022 article [0] says it generates binaries that run about 7% slower than what GCC produces with -O1.
[0] https://www.cs.cornell.edu/courses/cs6120/2022sp/blog/compce...
I wouldn't say so. The gap between GCC and Clang/LLVM is bigger than that on some benchmarks, [0] although the gap between their overall performance (using geometric mean) is much less. [1]
It's a very impressive result considering how much effort has gone into GCC. It highlights how compiler optimisation work can face diminishing returns.
> the norm for releases being O2 if I understand correctly
Agreed, but the difference isn't that great. From the article:
> CompCert code is 7% slower than gcc-01 and 12% slower than gcc-02 on average.
Reminds me of how the cproc/qbe C compiler gets very roughly 50% of the performance of Clang/LLVM in 0.1% the lines-of-code. [2]
[0] https://www.phoronix.com/review/gcc-16-vs-clang-22/3
[1] https://www.phoronix.com/review/gcc-16-vs-clang-22/6
[2] https://drewdevault.com/talks/qbe/ (and see the linked slideshow PDF, as the abstract is a little misleading)
I agree that this is an impressive feat. I also don't see why an equivalent for rust would be adopted.
A 12% difference in performance between two compilers, for some given application, is possible even between two of the best optimising compilers available. A 12% performance lag does not necessarily relegate a compiler to obscurity.
The long-entrenched MSVC compiler is still a major player despite that, for at least some applications and for at least some compiler versions, Clang outperforms it by far more than 12%. [0][1]
If a new Rust compiler really could offer significantly fewer compiler bugs, that would of course need to be weighed against any performance penalty, and the proper tradeoff would of course depend on the project.
> I agree that this is an impressive feat. I also don't see why an equivalent for rust would be adopted.
I imagine it wouldn't be widely adopted for most applications. CompCert itself is pretty niche. It's not Free and Open Source software, which I suspect is a factor, alongside its technical drawbacks.
Unless you're working on ultra-low-defect software, such as in a safety-critical domain, compiler bugs are unlikely to be a major factor in your software's quality. If the compiler lacks support for interesting language features, it becomes an even harder sell for 'ordinary' projects that are able to tolerate an ordinary number of compiler bugs.
[0] https://lemire.me/blog/2023/02/27/visual-studio-versus-clang...
[1] https://discuss.python.org/t/using-clang-cl-for-better-perfo...
Yes, that’s an incomplete and tenuous belief, but that’s where the irony is. If you’re omniscient you don’t get to experience irony.
I'm firmly in the "AI is incredibly powerful and useful" camp myself, but there's no question that currently, without good human oversight and feedback, AI output can have many issues which can be described as slop.
But yes, improved type systems are one of the ways this can be managed, and essentially help bootstrap more reliable systems.
There's a minor irony in imperfect system being used to help build more perfect systems, but you could say the same thing about human developers.
I used to compare this hype to crypto hype, but its so much worse. Can we just try not to shove llms into every single conversation.
Is there a hackernews type space where the word is banned? Even lobste.rs has this problem. I need some sort or way to just hide all content on certain sites where LLMs/AI are mentioned.
Like it or not, programming will become largely LLMs. And the comment you're replying to is actually making a pretty good case for where the use of LLMs is good: generating code to be formally verified.
Programming will never be dead.
1. Because the situation you describe is fantasy.
2. Even if it is a reality and all software can be verified (it cant), I will still be programming and so will millions of others because its fun, programming and sex are really damn close to things I cant live without. Not everyone got into engineering for the money. I legitimately love this activity, it is my actual passion. I would rather be poor than give it up.
Its like im being told everyday all day, non stop that the thing I love is not worth loving.
Maybe tech isn’t for you if you don’t like trying to stay on top of it.
Anyways, I probably know more about llms than you, I probably use them more than you. I just dont want to turn every conversation into a discussion about them.
I would argue the smartphone is 1000x more impactful than LLMs, along with many other inventions since the internet. I'm not really sure what contribution llms have had that are actually THAT impactful for peoples lives. Its possibly (but unlikely) a net negative, time will tell. Most people using them are not getting smarter, thats concerning. Most are just making more money for their bosses who are itching to replace them with an llm.
If we didn't let people live off the work of others, as the capitalists do, then we could all rejoice in that we've found yet another way of reducing the labor required to produce stuff.
Yet, in the real world, it seems as despite the progress of technology, the labor of the everyman is constant, and only the richest get more and more shiny jets and yachts.
The truth is LLMs is what you need to be optimising around as they will become the standard.
I'm not talking about mission critical software, but bespoke business applications that are produced by software engineering sweatshops and where the actual users are inured to incomplete features with sharp edges, arcane bugs, and incomprehensibly long intervals between bug report and fix.
Even in mission critical software we will likely see a lot of LLM tool usage. Even if the copyright plug is pulled and all vibecoded applications are regarded as copyright violations, its uses to analyze existing code and give advice will remain. In this world, writing applications entirely by hand will become an artisanal discipline.
Nothing is stopping you from enjoying the sort of programming that you like. But I think it's healthy to accept that that style will soon not be mainstream. It will be like knitting your own clothes, and that's okay.
Regarding formal verification, it's not needed, as LLMs already make on average about as many errors as the average human programmer.
For me, 90% of programming is just a tool to get the job done. It's like a machine in a factory that spits out 50 spoons a minute, and it doesn't need to be any more beautiful than that. If you enjoy woodcarving and want to spend an hour making your own spoon, do that and don't let the existence of the machine make you feel that hobby is any less worth loving.
I think that’s a myth! What tends to happen in mature software is there are unfixable tradeoffs that can be made that favor one use case or another. And of course people will keep adding features too
If your spacebar heater relies on such behaviour Rust can and will break it. If you're in the crater group you'll be notified and urged to upgrade it.
It will be interesting to see if fixing this bug actually causes any crates to not compile.
struct Q<T: 'static>(Option<T>);
impl<T: 'static> Drop for Q<T> {
fn drop(&mut self) {
// Code in here has access to a non-'static &String
// but assumes that it's a &'static String
}
}
fn main() {
let l = String::new();
let mut q = Q(None);
q.0 = Some(&l);
}
Most LLMs would probably say this piece of code won't compile.
It uses the same code, so yes.
(Although I don't know what are their guidelines, it might be that they require avoiding code relying on implied bounds, for example).
"q.0" is not really needed, it's a struct with unnamed fields. Here is the same with named fields:
struct Q<T: 'static> { pub x: Option<T> }
...
let l = String::new();
let mut q = Q { x: None };
q.x = Some(&l);q.0 is the first field of q, which is anonymous. They could have just as well called it x and written q.x.
struct Q<T> { value: Option<T> }
with a named field or you can define it like
struct<T> Q(Some<T>)
which is the same struct layout but without an explicit name for the value. In that case, the field is implicitly names `0`
So q.0 = Some(foo) is equivalent to q.value = Some(foo).
The Some(..) is just one variant of the Option enum:
enum Option<T> { None, Some(T) }
Option represents a value that may or may not be initialized. For uninitialized values you use the None variant and then you can initialize it to Some(value).
The C rough equivalent would be
struct Q { void* value; }
struct Q q = q { NULL }; q->value = ...
$ cat main.c
#include <stdlib.h>
#include <string.h>
struct Q {
/*@null@*/ /*@only@*/ char * t;
};
/*@only@*/
struct Q *
Q_new (/*@only@*/ char * t)
{
struct Q * q = malloc (sizeof *q);
q->t = t;
return q;
}
void
Q_set (struct Q * q, /*@only@*/ char * t)
{
free (q->t);
q->t = t;
return;
}
void
Q_free (/*@only@*/ struct Q * q)
{
free (q->t);
free (q);
return;
}
int
main (void)
{
/*@only@*/ char * l;
/*@only@*/ struct Q * q;
l = strdup ("");
q = Q_new (NULL);
Q_set (q, l);
free (l);
/* This is UB */
Q_free (q);
return EXIT_SUCCESS;
}
$ splint main.c
Splint 3.1.2 --- 21 Feb 2021
main.c: (in function Q_new)
main.c:15:3: Arrow access from possibly null pointer q: q->t
A possibly null pointer is dereferenced. Value is either the result of a
function which may return null (in which case, code should check it is not
null), or a global, parameter or structure field declared with the null
qualifier. (Use -nullderef to inhibit warning)
main.c:14:17: Storage q may become null
main.c: (in function main)
main.c:43:6: Unrecognized identifier: strdup
Identifier used in code has not been declared. (Use -unrecog to inhibit
warning)
main.c:44:13: Null storage passed as non-null param: Q_new (NULL)
A possibly null pointer is passed as a parameter corresponding to a formal
parameter with no /*@null@*/ annotation. If NULL may be used for this
parameter, add a /*@null@*/ annotation to the function parameter declaration.
(Use -nullpass to inhibit warning)
main.c:47:8: Dead storage l passed as out parameter to free: l
Memory is used after it has been released (either by passing as an only param
or assigning to an only global). (Use -usereleased to inhibit warning)
main.c:46:12: Storage l released
main.c:12:1: Function exported but not used outside main: Q_new
A declaration is exported, but not used outside this module. Declaration can
use static qualifier. (Use -exportlocal to inhibit warning)
main.c:17:1: Definition of Q_new
main.c:20:1: Function exported but not used outside main: Q_set
main.c:25:1: Definition of Q_set
main.c:28:1: Function exported but not used outside main: Q_free
main.c:33:1: Definition of Q_free
Finished checking --- 7 code warnings
I originally didn't do this, because I didn't feel the need to invoke UB without necessity. Also I wanted to demonstrate, that the error can (and is) caught in C, without actually committing the double-free (so that the program behaviour is still well-defined, but the abstraction is already violated).
Also, the Q_free function should not free q->t. It may access it (e.g. printing it), but it does not own that reference and is not responsible for deallocating its memory.
Sorry, I should have looked closer the first time! My excuse is that it was too early in the morning to be thinking about such things.
> Also, the Q_free function should not free q->t.
Well, that's because I thought it would own it. But if in Rust it is not implied that the Q owns the string, then I fail to see where the error is? Both l and q will be deallocated at the end of the main function. Depending on the order of the deallocation maybe q exists for a theoretical time while pointing to already deallocated memory. But both get deallocated at the same point, so q does not outlive the existence of l, so where is the error?
Why?
But I think https://news.ycombinator.com/item?id=50072252 answers it:
A 'static lifetime constraint on T says: types that satisfy T must not contain any borrowed value/content whose lifetime will be shorter than the life of the program.
The issue is that the destructor of q can observe a reference which is about to be deallocated as having a 'static lifetime (i.e. it will live forever), and can thus stash it away somewhere for other code to access after it's been freed (as in the examples in the issue which demonstrate UB.)
I gave it a quick scan for Rust examples; there are 11 samples among the following pages:
https://counterexamples.org/eventually-nothing.html
https://counterexamples.org/mutable-matching.html
https://counterexamples.org/selfishness.html
https://counterexamples.org/scope-escape.html
https://counterexamples.org/nearly-universal.html
I wonder if TFA's example could be reproduced without mutability.
EDIT: I said new features of the type system and more importantly, their interactions with the ever growing list of other features, not new features in the compiler. Totally different things. The compiler could be perfectly implementing the wrong choice of algorithms in the first place.
But the unsoundness is because the type system itself, abstractly, was not verified. Or the interactions between parts. I.e. If you verified the compiler you'll find that it implements the algorithms perfectly. But tells you nothing about whether those implemented algorithms, however perfectly implemented, actually provided those guarantees.
Would it be nice if we had even higher level of quality? Yes. But what’s neat is we have empirical evidence - Rust code does achieve the memory safety goals over C++ even though there’s bugs in the compiler. That’s why Rust is very quickly eating all things C and C++. A huge amount of value that the industry would just be unable to access for a long time if you waited for everything to be formally proven.
Rust cannot fail, it can only be failed.
Sweet. But why should anyone care..
You can't do that with Rust because "Rust's type system" is literally just the vague concept of "a type system that is safe". The implementation of this ideal in rustc can have flaws, but Rust itself cannot, because Rust itself has no flaws by definition.
You seem to be implying that because of this, Rust has no definition of its type system (other than whatever happens to be implemented in the current compiler). I would disagree.
As evidence, I would like to point out that not a single person argued that this flaw was not a bug, but was in fact the officially defined behavior. Clearly, the community has a shared understanding of what Rust's type system "ought" to do.
This is a nice ideal, but it doesn't actually exist in concrete terms as a design. It can never have flaws because the definition of it is just "rustc without all the flaws".
“Please build the perfect system. Because I specified ‘perfect’ any flaws are due to your incomplete implementation, not my awful brief.”
If the person writing the rust spec was a client I would have to drop them because they would be setting themselves up for dissatisfaction.
Also, a lot of those soundness issues aren't related to new features _at all_, but usually due to poor assumptions made in optimizations.
The soundness bugs _are_ taken seriously though, and are slowly being patched out. A lot of the links in OP are now closed because Rust either changed how it generates code to avoid the issues, or fixed LLVM.
hm, there's an effort but scoped only to the std
It's terrifically complex: https://doc.rust-lang.org/beta/nightly-rustc/rustc_middle/ty...
- people still get value off of an incomplete implementation, because we are allowed to be mindful of the issues (see, eg, UB in C++). In other words, many, many more bugs have been prevented by the borrow checker working as intended than have been caused by it working poorly. It's not particularly close.
- theory almost always moves slower than practice
The mutable state situation pointed out by the grandparent poster was a famous issue in typing SML under mutability that required serious changes in the language, see Section 4.2 here: https://smlfamily.github.io/history/SML-history.pdf
These were Turing-award caliber people figuring it out and, according to your definition, "yolo"ed it. This stuff is _hard_, and these driveby comments truly aren't helpful.
There are pros and cons to tests, but they do some things very well. For starters a test is a lot better sign that things work as intended than a formal proof which is often so abstract most people can't be sure how it relates to what was intended. You can prove something correct that is clearly wrong. While a test will only test the one case, at least it is typically easy to understand that case and why that answer is correct. Of course if your requirements are correct your proof will tell you it is correct for all cases while a test is limited to that one case. Again pros and cons, and people speaking against tests are not honest about those.
1) Unsoundnesses in Java's type system are checked at runtime.
2) They (or at least some) are intentional or at least won't-fix bugs.
Both are not true for Rust.
Is this more an abstract concern over the state of the industry, or is there a specific language you had in mind that you'd prefer Rust was modeled after?
It is moral though, so who cares really.
Safety is a nice side-benefit. The main reason why I like the safety is because it allows you to fearlessly write high-level code without being constantly paranoid about memory as in C++. It’s more about enabling ergonomics without loosing performance, rather than security or correctness.
The real shortcoming it is very common for people to not write modern C++. I know several C++ developers who refuse to use smart pointers because they don't like magic happening.
C++ has a terrible reputation which is well associated with, say C++98, if you have a good linter you can bop people on the head for using news anywhere outside a few limited locations where you consider it warranted. All working environments have language features or even programming paradigms that are considered detrimental to code ecosystem health and fighting against these is an organizational lift when working in any programming language. Full type security doesn't mean someone can't create an overly generic object and just start breaking code contracts across the code base. C++ has an awful lot of footguns and more warts than most but it can be made to be sane for development and the new language features really bring it on par with other modern languages.
I have worked in C++ (primarily within the Qt framework which pushed hard into const refs instead of pointers) and it's fine. I've also worked in PHP which can also be fine. Rust isn't something I've worked at scale in but it certainly is very attractive if I ever jump to an org working in it - but even in Rust you'll want to restrict style divergence and bad patterns with linting.
std::unique_ptr<int> p(foo());
std::print("{}\n", *p);
These issues will exist for all time. The only question is how rare can we make them.
The obvious follow-up is to find a minimal set of rules that allow us to express what we want without permitting temporal unsafety. The borrow checker is one set of fairly minimal rules that works pretty damn well. Alternatively you could define away the unsafety by giving it semantics, like fil-C does.
It is kind of annoying to use, but at least there is an obvious default choice.
If I need the absolute fastest code, then that code will be tortured under valgrind. That’s non-negotiable in my book.
Code is not done until its memory management is proven correct.
If not, why would they be allowed to do the same in C++
With access to inline asm, all speed is ultimately more a measure of effort applied than PLang. So those perf ideas are just "with my level of effort". Even Python via Cython has pretty direct access to asm. The "do it in pure X" that Sun popularized (even in a meta-/cross-PLang-sense) as a business attempt to define Java virtual machines as The platform has had many casualties, not the least of which is the quality of discussion of PLang practicalities, not just on HN, but everywhere.
PS: I was very enthusiastic about Cython for a good while and I went deep into it, but I found that it is rather a mess to use in practice. Yes you get C performance within Python, kinda, but you also get all the C footguns, amplified by the complex interactions with Python features.
EDIT: With Cython how I mostly kept things organized was the "caller allocation" so common in C where memory was always managed the usual mostly refcounting way of CPython/NumPy, but I had dozens (maybe even 100) little "kernels" I could pass that memory into that could even use Intel/MSFT "intrinsic" asm calls (things like immintrin.h) or even just #include C code which did similar. It didn't seem too bad to me, but Nim always seemed a more complete solution (at the PLang level - not counting ecosys). (And these days there is even NimPy (https://github.com/yglukhov/nimpy) to write low-level speed-up code for Python in Nim.)
Yes, you can pass around an object of procedures, but that is awkward, doesn't have any support from the language or stdlib, and from what I can tell isn't a normal thing that is done.
I think you & I are really back & forth-ing "the PLang not enforcing/strongly encouraging a very specific way of doing things that a whole ecosystem/community organizes around". Relying instead on self-organization has many pros & cons, to be sure. Lisp suffered much the same "until" Common Lisp (which still leaves many things open). My tag line lately is: delegation affords so much, but trust sure is tricky. Much of the time people seem to discount the often rather large cost of establishing/maintaining trust. (I'm not saying you are - just that it's an easy thing to do. I think it's why there is this is a big megathread on "trusting rust", even though "rust" is a veritable suffix of "trust". Lol.)
I'd disagree with this point, there is more typing involved with a first draft of code than Python or Typescript, that's certainly true, but if we bring all the tools (e.g. docblock comments for "type declaration") that are habitual to make up for the shortcomings in most developer's minds then that gap shrinks and, imo, ends up swapping. It's cheaper to have automatically enforced and consistently maintained checks than moving that information out of band and the resulting code in a strictly typed and memory enforced language ends up removing a significant cognitive overhead when reading and maintaining the code.
I have a strong aversion to boilerplate code so it's important to try and allow terse expressions when possible - but unstated assumptions are worse than overstated assumptions.
Consider reading past the hashtag.
The hand-waving comes from other languages which can’t do anything to prevent their users encounter such bugs and the ensuing CVEs.
But of course Python function were always duck typed so it often didn't matter to the user.
Typing prevents 1 minor bug per developer per year reaching the testing environment.
It costs 2x development time to use. (With LLMs it's doubles token usage, very measurable)
It's not actually a sane trade-off.
I've found type hints to be remarkably useful and catch significantly more bugs than I ever would have thought, and have quickly picked up the practice of using them wherever I code. My general finding is that if I have to struggle to put useful type hints in place, I probably need to rethink my code.
I even type hint my unit tests because I've learned that sometimes type incorrectness means my tests haven't been testing what I thought they were testing.
Frameworks like pydantic[0] can help you turn those type hints into something runtime enforcing too.
[0] https://pydantic.dev/docs/validation/latest/get-started/
As soon as static typing is introduced into a Python code based, the developers start writing unscalable coding patterns that don't scale to larger programs.
Even beyond Python, most gradual type systems are unsound by design.
A few past discussions if anyone is curious:
Counterexamples in type systems (2021) - https://news.ycombinator.com/item?id=48718972 - June 2026 (4 comments)
Counterexamples in Type Systems: programs that crash, segfault or explode (2021) - https://news.ycombinator.com/item?id=36213704 - June 2023 (97 comments)
Counterexamples in Type Systems - https://news.ycombinator.com/item?id=27252899 - May 2021 (62 comments)
The report seems to claim the opposite:
> Looking at the bug itself, it is also very likely that this unsoundness has not caused memory unsafety by accident. While it looks easy to trigger, to exploit this you need an intentionally malicious Drop implementation.
I guess my take is that I'd rather assume this is exploitable in the wild until sufficient time has passed. Would be cool to see clippy pick up on this bug in older versions!
It is, but safe Rust merely makes it hard to do by accident. Leaking is not "memory unsafe" because is does not currupt anything, so it's legal.
This bug wouldn't be nearly as interesting if it "just" resulted in a memory leak. Memory leaks aren't considered unsafe in Rust because they can't result in UB in the same way as e.g., OOB writes. In fact, there are APIs accessible in safe Rust which explicitly leak memory! [0]
This bug is effectively a use-after-free, which is definitely something that the Rust compiler should catch.
> Would be cool to see clippy pick up on this bug in older versions!
No need for that. A future version of the Rust compiler that incorporates a fix should reject the incorrect code, so all you'd need to do is compile older code with that new compiler.
Possibly. I'm no rust expert, so I had the read the code slowly, building up a mental model of the types being used. For me, the effort that required is one of the more painful aspects of Rust.
Nonetheless when I got to this line, I stopped reading:
q.0 = Some(&l);
I write lots of code that doesn't compile of course. It's perfectly safe because the compiler catches it. Right now, I feel like a climber at the top of a mountain who just discovered he forgot to attach his safety harness.
Who knows. Maybe all of crates.io will compile with the fixed compiler, because this is a mistake humans or LLM's never make. We will see.
You don't need 'static to trigger the bug [0]
[0]: https://github.com/rust-lang/rust/issues/164250#issuecomment...
If that's true it's not a very complex or subtle bug. The very first response of "How on earth was this not found before this???" seems entirely appropriate to me.
Rust as a language is capable of that. If you don't use the standard library then you can provide that functionality yourself. The standard library is trying to support it, without creating interfaces that are a massive pain to use.
For instance, it's easy for us to support `Vec::try_reserve`, and we do, but that's not trivial for users to use everywhere; it's error-prone, and it's easy to make a mistake and call a method that goes past your reservation. It's painful for us and for users to make `Vec::push` support this (and having a `try_` variant for every function), so we've been working on a better solution. One current possibility is to have a dedicated type for a Vec that has OOM handling, where `push` can return an error, so you know if you use that type you can never accidentally fail to handle OOM.
Sure if you ignore the most popular desktop OS in the world.
Also overcommit (at least how Linux does it) is dumb. It leads to broken OOM behaviour (killing a totally random process, or more often locking up the system and hard rebooting). And some apps do need to handle OOM properly, e.g. formal verification systems which can use enormous amounts of memory. Jasper has to resort to monitoring page fault stats to know when it is using too much memory.
no it doesn't. the broken oom behavior is distros not configuring earlyoom/systemd-oom properly. The problem here is that the kernel doesn't know what software is critical to your OS so the distro needs to be responsible for what processes to protect.
For apps that need to handle oom, Linux is missing something a handle to tell apps to shrink their memory use. Apple does this and it seems to work quite well (and proves that Linux style overcommit works well since it's what they use)
For no_std style work where running out of memory is actually a hard limit, you have two choices:
1. disable the global allocator and use APIs that take Allocator, to force yourself to only consume APIs that can return errors from allocation. (The ergonomics aren't always great, and fattening all types with a pointer to Allocator isn't great; might be able to use a zero-sized Allocator, with trickery. Say goodbye to many libraries.)
2. use static analysis and try to avoid ever hitting a code path like Vec::with_capacity that doesn't return a Result. Make sure the tooling also analyses your dependencies. (Tooling for the static analysis is painful. And still, say goodbye to many libraries.)
I do wish there were more conveniences for alloc-can-fail APIs, like the failable version of Vec::push needs both try_reserve and push_within_capacity (nightly only).
When implementing the type system, however, variance does have to be treated with care. Historically it wasn't much, now it is.
I'm using sqlite's testing principles that Dr Hipp explains (MC/DC, mutation testing, fuzzy checking, property testing) and even some loose lean proofs written by LLMs, but in the end, it's all just kayfabe. I'll find out some bug one day anyway.
I will happily continue to write things in Golang first. I don't enjoy refactors in Rust, and it's rare that I see things that genuinely need its raw performance.
Cloudflare's Oxy is a stellar example of build in Golang till your design is settled and then port over for performance when you need it.
From the GitHub thread: "I think unification just generally doesn't check lifetime bounds of the type." [0]
There are steps, or rules or whatever, that perform soundness (or well-formedness or whatever) checks. And clearly one was missing.
My guess was just that maybe someone didn't think it was necessary for soundness.
[0]: https://github.com/rust-lang/rust/issues/164250#issuecomment...
impl<T: 'static + Send> Drop for Q<T> {
fn drop(&mut self) { if let Some(t) = self.0.take() { KEPT.lock().unwrap().push(Box::new(t)); } }
}
struct Q<T>(Option<T>);
impl<T: 'static> Drop for Q<T> {
fn drop(&mut self) {}
}
error[E0367]: `Drop` impl requires `T: 'static` but the struct it is implemented for does not
--> src/main.rs:3:9
|
3 | impl<T: 'static> Drop for Q<T> {
| ^^^^^^^
|
note: the implementor must specify the same requirement
--> src/main.rs:1:1
|
1 | struct Q<T>(Option<T>);
| ^^^^^^^^^^^You also don't have to place those bounds on the Drop impl just because they're on Q, that's not an error.
And I'm still not any closer to understanding what the bug is about.
It doesn't even have to be less constrained than the containing types definition. You can provide an impl with more constraints; for example, a type Q<T: Read + Write>, but impl Seek for Q<T: Read + Seek>
Do you happen to know the specific rules of when it's an error and when it's not?
That said I was wrong earlier.
This is the problem when a language advertises with “guarantees” and it turns out it never had any due to many false assumptions by their promoters.
So what other soundness bugs are there in the language that we need to know about?
For some reason this is controversial, but it remains true.
But not custom drops that are doing variadic type stuff like this, usually.
- you've never written a custom Drop
- "Not really". Implying that your experience generalizes
I've written a bunch of rust programs without custom Drop. I believe your first statement. There are large classes of programs that don't need it. There are also many that do. I think your statement says a lot more about what kind of programs you work on than about custom Drop not being common. People are downvoting the implication that custom Drop is not a common pattern.
That it took 11 years for someone to come across this is a fairly intense counterargument to your attempt to paint this as a language with dangerous footguns.
The reality is that if this were actually code that people wrote, well, it wouldn't have sat around for a decade.
The fact that nobody reported this before means it's not, by definition.
A program isn't invalid just because it violates the ownership and borrowing rules of Rust. There are tons of programs in the world with shared, mutable access, devoid of Mutexes, Arcs and Locks that nonetheless are completely valid.
Maybe they wrote it in C :-)
On a serious note, old software is generally more trustworthy than new software, because bugs are discovered over time and with usage.
The age of software means bugs are more likely to be discovered but not necessarily fixed. In the case of Rust, the developers have documented a whole lot of unsoundness bugs that require a new borrow checker architecture to fix.
Having ideals does not mean there are never bugs.
With this you have to recompile everything
I’d be shocked if people working on and passionate about thing didn’t shit on other thing. That’s natural and even good in moderation. It’s up to a leader to keep it under control.
This is fine, for the reasons you state: I can use the language and tools while ignoring the jerks.
Moreover, I wouldn’t say they’re completely ignoring supply chain risks, as there have been discussions about this as early as 2021 [1] and there are ongoing efforts to address this [2]
[1] https://internals.rust-lang.org/t/about-supply-chain-attacks...
People who say this just dont know what they're talking about and are repeating youtubers and X takes.
There's a difference between people you disagree with and toxic behavior. Can you give examples of toxic behavior?