Stop making me memorize the borrow checker
erikmcclure.com
erikmcclure.com
There is a learning curve for everyone when they pick up a new language: You have to learn how to structure your code in ways that work with the language, not in the ways you learned from previous languages. Some transitions are easier than others.
There should come a point where the new language clicks and you don’t feel like you’re always running into unexpected issues (like the author of this article is). I might have empathized more with this article in my first months with Rust, but it didn’t resonate much with me now. If you’re still relying on writing code and waiting for the borrow checker to identify problems every time, you’re not yet at the point where everything clicks.
The tougher conversation is that for some people, some languages may never fully agree with their preferred style of writing code. Rust is especially unforgiving for people who have a style that relies on writing something and seeing if it complies, rather than integrating a mental model of the language so you can work with it instead of against it.
In this case, if someone reaches a point where they’re so frustrated that they have to remember the basic rules of the language, why even force yourself to use that language? There are numerous other languages that have lower mental overhead, garbage collection, relaxed type rules, and so on in ways that match different personalities better. Forcing yourself to use a language you hate isn’t a good way to be productive.
I could've expressed the sentiment in this blog post back when I started playing with Rust ~2016. Instead, I ended up learning why I couldn't pass a mutable reference to a hashmap to a function I'd looked up via that hashmap (iterator invalidation lesson incoming!).
The kind of bug I was trying to add exists in many languages. We can only speak in general terms about the code the blog post is talking about, since we don't have it, but couching it in terms of "doesn’t fit their preferred way of coding" misses that the "preferred way of coding" for many people (me included) involved bugs I didn't even realise could exist.
Any example except C++? BTW, the closest thing possible in C# is modifying a collection from inside foreach loop which iterates over the collection. However, standard library collections throw “the collection was modified” exception when trying to continue enumerating after the change, i.e. easy to discover and fix.
This is exactly what the mutable map pointer was for, for the function to be able to modify the collection; C++ would result in potentially iterating garbage, C# it sounds like would throw an exception (and so show the design wouldn't work when I tried to test it), Python definitely didn't do a graceful thing when I tried it just now. And if I had a collection struct in C, I'm sure I could've done some horrible things with it when trying.
The best of those outcomes is C#, which would've shown me my design was bad when I ran it; that could be as early as the code was written if tests are written at every step. But it could be quite a bit later, after several other interacting parts get written, and you rely on what turns out to be an invalid assumption. For Rust, the compiler alerted me to the problem the moment I added the code.
FTR I ended up accumulating changes to the map during the loop, and only applying them after the loop had finished.
EDIT: Python did do something sensible: I didn't expect pop'ing an element from the list to echo the element popped in the REPL, and got a printed interleaved from front to back, which does makes sense.
Java for sure.
> However, standard library collections throw “the collection was modified”
Java is similar, but the exception is done on a "best effort" basis, and is not guaranteed.
C++ will laugh in invalidated iterators.
Of course you can erase from the container you are iterating over, but you have to make sure to continue the iteration using the iterator returned from the erase function and avoid storing copies of .end()
This just seems to be the standard excuse I read any time someone has a critique of Rust.
The guys who parrot this blend of comments don't seem to be aware that cognitive load is a major problem, not a badge of honor.
With c you also need to understand pointers, manual allocation, volatility, is that bad?
And how did we get from that to “for some people, some languages may never fully agree with their preferred style of writing code”?
If a C++ programmer (ideal Rust learner) says four years in that the ergonomics of Rust are bad, then based on my own experience I will believe them.
I don’t write something to see if it compiles either, I design every single line of code. But with Rust (and a lesser extent C++) a lot of that design is for memory safety and not necessary for the algorithms.
> If a C++ programmer (ideal Rust learner)
There is no ideal learner.
It's high time that people in the Rust community such as you just stop with this act.
The Rust community itself already answered that they find "Rust too difficult to learn or learning will take too much time" as a concern to not use Rust. The community also flagged Rust being too difficult as one of the main reasons they stopped using Rust.
https://blog.rust-lang.org/2024/02/19/2023-Rust-Annual-Surve...
So building libraries must be easy, that's totally why you can become an architect over night ... What a stupid argument you delivered
I'm not sure you're paying attention. The people who are saying Rust is too hard are the Rust community itself. They said so in Rusty's annual survey. The ones who participate in it are Rust users who feel strongly about the topic to participate in their governance.
It's Rust users who say Rust is too hard. There is no way around this fact.
I agree completely. But I'd reword your statement slightly as "Anyone who has managed to become proficient in the language that is notorious for being one of the most complicated and convoluted languages in existence, with the most footguns per line of code and a CVE record that proves its difficulty, ..."
It changes the conclusion just a tad.
Saying it is easy to learn is just delusional and does a disservice to the language. Rust has many advantages, but trying to get people learning the language by lying about why they should learn it is just dumb.
Replying to that with "Rust is easy to learn" just makes it sound like you didn't even understand what you're trying to reply to.
No one really claimed it was easy to learn. Find me one Rustacean that claims Rust is as easy as Python/Ruby/Go.
But it is higher level language with decent ergonomic and in such way it can be interpreted as user-friendly.
> If a C++ programmer (ideal Rust learner)
If Java programmer (me) can learn Rust in a year enough to contribute to Servo development, you (ideal Rust learner) should have no problem either.
Plus Google saw about 6-month to get people up to speed with Rust.
It's a great example of unintended comedy the fact that the comment right below yours is literally "Rust is easy to learn and user friendly."
I can see user friendly. It's got a solid ecosystem around it, and compiler is super helpful with errors.
But easy to learn it ain't.
Well there's your problem. Rust does look like a semi-colon language, that's intentional, but if that's all you understand you're probably going to struggle.
The "ideal Rust learner" would have an ML, such as Ocaml or F#, maybe some Lisp or a Scheme something like that, as well as a semi-colon language like C or Java not just the weird forced perspective from C++
One experiment I probably won't get to try is to teach Rust as First Language for computer science students. Some top universities (including Oxbridge) teach an ML as first language, but neither teaches Rust because of course Rust is (relatively) new. The idea is, if you teach Rust as first language your students don't have to un-learn things from other languages. You can teach the move assignment semantic not as a weird special case but as it it really is in Rust - the obvious default assignment semantic. I pass a Goose to a function, obviously the Goose is gone, I gave it to the function. Nobody is surprised that when they pass a joint they don't have the joint any more, so why are we surprised when we pass a String to a function and the String is gone now? And once we've seen this, the motivation for borrows (reference types) is obvious, often we don't want to give you the string, we just want to tell you about a string that's still ours. And so on.
This is a silly point to make. What makes a C++ programmer a C++ programmer is not an uncanny ability to find a semicolon on the keyboard. It's stuff like using low-level constructs, having a working mental model of how to manage resources at a low level down and how to pass and track ownership of resources across boundaries. This is not a syntax issue.
It's absurd. You have people claiming that Rust is the natural progression for C++ programmers because their skillsets, mental models, and application domain overlap, but here are you negating all that and try to portray it as a semicolon issue?
So this means there's a substantial learning curve, or, if you try to just keep writing C++ even though you're in Rust, an impedance mismatch. It looks superficially like the thing you're used to, but that's not what it is.
† In Rust the semi-colon is turning your expression into a statement, but in the semi-colon languages it's a separator, everything is a statement anyway. So while a rust program might say let x = if k > 3 { plenty(k) } else { too_few() }; in the semi-colon languages there's a whole separate ternary operator provided to do this trick - in fact they don't have any other ternary operators and so they often call this "the" ternary operator which is very funny if you come from a language which has the multiply-accumulate operator...
The comment made sense to me: it's not at all about semicolons, but the semicolon isn't a bad marker to distinguish between language families. It's closely related to expression- vs statement-focused languages, which if you squint is similar to the distinction between imperative and functional languages. Though the latter distinction is less relevant here, I'm just using it to point out that semicolon syntax is suggestive of semantics, even if it is itself just a superficial bit of syntax.
You are correctly describing the awareness of resource management as an important characteristic, but that resource management is heavily intertwined with how values flow through the language syntax, which brings us back to expressions vs statements.
This is why I call C a minefield.
On that note, C++ has such an explosion of UB that I don't generally believe anyone who claims to know C++ because it seems to me to be almost infeasible to both learn all the rules, or at least the subset required to be productive, and then to write/modify code without getting lost.
With rust, the amount of rules I need to learn to understand rust's borrow checker is about the same or even less. And if I forget the rules, the borrow checker is there to back me up.
I still think that unless you need the performance, you should use a higher level language which hides this from you. It's genuinely easier to think about.
That being said, writing correct rust which is going to a: work as I intended and b: not have UB is much less mentally taxing, even when I have to reach for unsafe.
If you find it more taxing than writing C or C++ it's probably either because you haven't internalised the rules of the borrow checker, or because your C or C++ are riddled with various kinds of serious issues.
Computing is a series of "minefields." At least you get a map of this particular one.
I'm far more confronted by public facing APIs that involve user authentication than I am of any particular documented set of language facts.
There has been a proposal to attempt this for C++ but IMO the progress on making such an appendix is slower than the rate of the change for the language, making it a never ending task. It was also expanded by the fact that on top of Undefined Behaviour C++ also explicitly has IFNDR, programs which it declares to be Ill-formed (ie they are not C++) but No Diagnostic is required (ie your compiler doesn't know that it's not C++). This is much worse than UB.
For the rest of us, there’s cppreference, UBsan and quite a few books on writing correct C++ code. Of course, these will still not suffice to write 100% memory safe code, which is a pretty arbitrary goal that just happens to match what Rust offers and is pushed a lot by Rust advocates.
It’s a nice goal, but not everybody works on software that’s attacked all day every day.
I'm extremely positive on highlighting as many of these problems before it gets to the user's hands, even more so if it's as early as a compile step as in the borrow checker, but lets not delude ourselves that they are the only possible issue software has. Or that in many languages it's a tooling issue (or culture issue accepting that tooling...) rather than a fundamental language difference.
On a side node, with the prevalence of things WASM I feel some people are just redefining what "memory safety" is. Defining a block of memory and using offsets within that is just reinventing pointers, the runtime ensuring that any offsets are within that block just mirroring the MMU and process isolation. We should really be looking at why that isn't well used rather than just reimplementing a new version on top for "security", as if those reasons aren't really "technical" (IE poor isolation between "Trusted" and "Untrusted" data processing in separate processes due to it being "Easier") we need to ensure we don't just do the same things again, and if they are technical we can fix them.
1. Masochism
2. Underspecification, in a vain attempt to make a language that can theoretically be used on PDP computers.
3. The fact that an inappropriate write through a pointer results in behavior that is so undefined that it can lead to remote code execution and hence do literally anything.
No amount of additional specification can fix #3, and masochism cannot explain it.
One could mitigate #3 to some extent with techniques like control flow integrity or running in a strongly sandboxed environment.
This is a strange way to look at it. You'd get remote code execution only if the result of writing through the pointer was exactly what you'd expect: that the value you tried to write was copied into the memory indexed by the pointer.
I'm talking more about the nonsense like "c++ + ++c". There's no reason but masochism to keep it undefined. Just pick one unambiguous option and codify it.
An example of #2 is stuff like signed overflow. There are only so many ways to handle it: wraparound, saturate, error out. So C should just document them and provide a way to detect which behavior is active (like it does with endianness).
> I'm talking more about the nonsense like "c++ + ++c". There's no reason but masochism to keep it undefined. Just pick one unambiguous option and codify it.
It's because there's an underlying variance in what the compilers (and the hardware [1]) translated for expressions like that, and codifying any option would have broken several of them, which was anathema in the days of ANSI C standardization. (It's still pretty frowned upon, but "get one person to change behavior so that everybody gets a consistent standard" is something the committees are more willing to countenance nowadays).
> An example of #2 is stuff like signed overflow. There are only so many ways to handle it: wraparound, saturate, error out.
Funnily enough, none of the ways you mention turn out to be the way it's actually implemented in the compiler nowadays.
As for why UB actually exists, there are several reasons. Sometimes, it's essential because the underlying behavior is impossible to rationally specify (e.g., errant pointer dereferences, traps). Sometimes, it's because you have optimization hints where you don't want to constrain violation of those hints (e.g., restrict, noreturn). Sometimes, it's erroneous behavior that's hard to consistently diagnose (e.g., signed overflow). Sometimes, it's for explicit implementation-defined behavior, but for various reasons, the standard authors didn't think it could be implemented as unspecified or implementation-defined behavior.
[1] Remember, this is the days of CISC, and not the x86 only-very-barely-not-RISC kind of CISC, the heady days of CISC where things like "*p++ = --q" is a single instruction.
In embedded environments you're constrained by toolchain and platform but it's still a bad idea to rely on any behaviour which your compiler doesn't provide a definition for (which might be more behaviour than what your standard provides a definition for) because changes to the version of the compiler or even changes to surrounding code can trigger issues caused by reliance on UB.
It's not actually that hard to write embedded code which does not invoke UB outside of register access and even there it's possible to limit yourself to invoking behaviours which the combination of hardware + compiler does provide documented behaviour for.
(source: I've written embedded code which did not knowingly/intentionally invoke UB outside of register access and in those cases the implementation did define behaviour.)
Exporting and consuming the full c abi with very little effort is also another huge thing in rust's favor. Languages have opted heavily for supporting calling into the c abi and being hosted by the c abi, so naturally support for rust on the same terms comes for free. There's even rust in linux now.
Learning the rules only partly mitigates this, because sometimes one does exploratory programming and isn’t sure what the final types are or they just want to change something.
Rust thrives on over-specification which calcifies the APIs.
Anyway, just as the author’s allegedly holding Rust wrong, one could say that you’re holding C++ wrong - the right approach is to learn how to write correct code and then the exceptions. Also accept and be at peace with the fact that your code will have some bugs. I don’t know why the average Rust developer is so obsessed with getting things perfect and no less with memory safety when the overall software quality is the way it is. I mean if someone’s researching the topic or works on Rust, sure, be the Stallman of memory correctness.
Moreover, it becomes increasingly unpleasant and unworkable to deal with code which progressively gets more and more unreliable.
It's expected that if the complexity of a program grows, the state space that the program can occupy grows with it. But with UB you can run into by accident that state space seems to grow exponentially in comparison to a language like Rust.
If you are required to write code at that low level, I would not use anything other than something like rust.
If you are not required to write code at that level. There are many languages with much less uncertainty than C++ which are much more productive than either C++ or rust.
I think that's too strong a statement, because it applies to in-development programs. I agree with you if you're talking about released programs, but there can be benefit in leaving open the possibility of detectable flaws, serious or otherwise, while your code is still in development.
It's analogous to only compiling and running in debug mode throughout your development, and then switching to release mode for the final binary. The binary is suboptimal throughout your development process; it's too slow. But as long as the `--release` flag doesn't require any code changes, it's still a better idea than developing entirely in release mode.
Similarly, the binary could be suboptimal from a correctness standpoint, as long as removing the `--devel` flag only works when the compiler is fully happy. `--devel` could turn some borrow checking failures into warnings and still give you a runnable binary. Or it could allow leaving types underspecified in interfaces, and do an unsound type inference. Best case, it could even do runtime checks and/or coercions to establish the assumptions that the callee was compiled with.
Whether it would be worth the complexity is an open question, but it seems reasonably clear that Rust has a problem with brittleness to development-time change.
There currently exist no tools which with complete reliability point out all UB in your program. If any part of your program can have UB and you didn't write it with the explicit intention of not having UB in it at any point then you're going to be left with a tough situation to deal with.
I've read a lot of C in my time and there's codebases which I read and find easy and quick to review because they stick to the rules and only bend them sparingly and then there's codebases which are a pain to evaluate even the most basic parts for errors and UB.
There's no such thing as "detectable UB", there's only UB which your tools have luckily managed to detect.
Leave the UB to the people who can't avoid it, stick to safe languages when you can.
Why? What's wrong with using one of the many static code analysis tool to tell you about them if/when they appear?
I'm sorry, can you explain what leads you to believe your hypothetical scenario is an argument rejecting the use of static code analysis tools?
I mean, I'm stating the fact that there are many many tools out there that can pick up these problems. This is a known fact. You're saying that hypothetically perhaps they might not catch each and every single hypothetical case. So what?
It's a good thing no one made that claim, then.
The whole point is that were seeing people in this thread making all sort of wild claims on how it's virtually impossible to catch these errors in C++ even though back in reality there are a myriad of static analysis and memory checker tools that do just that.
Your average developer also knows how to type in a space character but still it's a good idea to onboard linters and automatic code formatters.
> Why? What's wrong with using one of the many static code analysis tool to tell you about them if/when they appear?
You clearly pose static analysers as an alternative to understanding UB. You still need to understand how things work.
Rust is really nifty, but there are still (many) things that could be improved in Rust, and we'd all benefit from more competition in this space, including Rust! This is not a zero sum game.
Honestly, I also think many people just want a nice ML-like with a good packaging story, and just put up with the borrow checker to get friendly C-like syntax for the Option monad, sum types with exhaustive matching, etc. This is a use case that could very much benefit from a competitor with a more conventional memory model.
The sentence in my original post that this addresses is supportive of the emergence of an alternative language to Rust for people with this use case, so I think we're just agreeing. (Although I wouldn't go so far as to tell others what they do or do not like based on my own ideas of what is essential and what isn't.)
I'd characterise it as a gentle criticism of the way the Rust community tends to react to anything other than effusive praise.
Rust is a nifty language, albeit with room for improvement, that falls into the (sadly overpopulated) category of 'neat thing, somewhat obnoxious fan club'.
It’s just that time after time I’ve heard people criticize Rust because they were, in fact, trying to write their pet language in Rust. It’s similar to how many complaints I’ve heard about Python because “it’s weakly typed”. What? Feel free not to like either of them, but make it for the right reasons, not because of a misunderstanding of how they work.
Now, the author of this post may be doing everything right and Rust just isn’t good at the things they want to use it for. The complaint about constantly bumping against the borrow checker leads me to wonder.
Are you sure you're not just being harsh to people whose grasp of CS vocab is weaker than yours? If someone tells me that Python is 'weakly typed', I translate it in my head to 'dynamically typed', and the rest of their complaint generally makes sense to me, in that the speaker presumably prefers static typing. Which is a valid opinion to hold, not necessarily the result of any misunderstanding.
However, I’ve heard lots of utterly wrong criticisms of Python (and Rust and…) that were based on factual misunderstandings and not just a vocabulary mistake.
I don't think that it is a very constructive article. The author's critique of Rust raises questions like "how to do it better" but there are not answers.
> they're always met with a wall of "you must not truly get the borrow checker,"
Yeah, it is frustrating in this case particularly. The author openly states that he doesn't want to learn all the quirks of the borrow checker, and people respond to it with "you just don't get the borrow checker". I can see how this answer could be helpful, but if it was expanded constructively, if there was an explanation how it can become easy to deal with problems the author faces if you understood the borrow checker. OTOH I cannot see how such an argument can be constructed without a real example with the real code and the history of failed changes to it.
I personally feel, that the borrow checker is simple, if you got it. And the author's struggles just go away, if you got it. You can easily predict what will happen if you try this or that changes to the code, and you know how to do something so the borrow checker will be happy. But I cannot elaborate and to make it clear how it works.
I'm far from a Rust expert, but to me if someone is whining about how it is hard to track lifecycle rules of an object because they are passing it through long chains of function calls across all sorts of boundaries, what this tells me is that you're creating your own problems that you could avoid if you simply passed the object by value instead of by reference. I mean, if tracking life cycles is a problem then why not prevent it from being a problem? Not all code lives in the hot path. I'm sure your performance benchmarks can spare a copy somewhere.
So you're suggesting that people should just wrap everything in Arc or make copies everywhere to avoid lifetimes? At that point why not just use Java/OCaml/Swift/your-favourite-GC-lang?
You're the only one who managed to come up with this nonsense. No one else did, and clearly you did not pick that from what I wrote because I definitely did no wrote that.
Please refrain from slippery slope fallacies.
you mean except the original poster often implying exactly that in their articles or the personal experience of the commenter that nearly always whenever they ran into borrow checker problems it was due to exactly that reason
> There's never anything to improve on Rust, it's always user error / a skill issue.
RFCs get accepted and implemented nearly every week, like in any language there is always a lot to improve
the problem is that the complains of such articles are often less about aspects where you can improve things but more about "the borrow checker is bad" on a level of detail which if you consider the borrow checker a fundamental component basically is "rust is fundamentally bad"
Maybe not! Maybe it’s truly just Rust being stubborn and difficult. However, it’s such an easy trap to fall into that I’ve gotta think it’s at least possible.
Edit: Oh! And use “cargo clippy” regularly. It makes excellent recommendations for how to make your code more idiomatic, with links to docs explaining why it’s nicer that way.
Thinking about your answer a bit more, one of the paradigms of Rust is “there shall be many immutable references or just one mutable reference” and so I can see that functional programming would naturally lead to that. But it’s a paradigm that works with the underlying principles rather than the true nature of the language, IMHO.
I do it by thinking about different domains of object graphs, and how data moves between them, for example.
1. No class hierarchies and inheritance.
2. The borrow checker forces a tree structured ownership style. You don't get spaghetti ownership. This is generally great because that coding style leads to fewer bugs. But sometimes it is annoying and you have to use indices rather than pointers as references.
Rust in particular is *really* obnoxiously bad at OOP patterns, and I think my lesson at this point is that this is because it is hard to do OOP safely, at least in a way that jives with its borrow checker. Something like functional core, imperative shell seems to be a much nicer flow for the thing in general.
Anyway, I've just got the one major Rust project (an NES emulator) so I'd say I'm pretty early in my Rust journey. For me personally, the good points (delightful match, powerful enum) outweigh the bad (occasional borrow checker weirdness, frustrating lifetimes) but I think it depends a lot on what you're trying to do with it.
Rust also pushes you to refactor in a way that really pulls out the core of your problem; the refactoring is just you understanding the problem at a deeper level (in my experience)
Oh boy. I see bugs everywhere in C and why the borrow checker exists. It really forces you to understand what happens under the hood.
The most issues in Rust are indeed related the expressions - you don't know how to describe some scenario for compiler well-enough, in order to prove that this is actually possible - and then your program won't compile.
In C, you talk more to the computer with the language syntax, whereas in Rust you talk to the compiler.
Any examples that you could provide? I have been dealing with C/C++ for close to 30 years. Number of times I have shot myself with undefined/unspecified behavior is less than 5.
I'm sure you've seen plenty of use-after-frees/use-after-move/dangling pointer type things or null pointer derefs, or data races, etc etc. These are largely impossible to do in safe rust.
Of course, I have done such mistakes, but they were caught early in the dev. process. I am talking about bugs that were caught in production due to misunderstanding of C compilers on 16/32 bit processors.
I also avoid idioms like *p instead write p[i] whereever possible.
[1]: https://doc.rust-lang.org/reference/behavior-considered-unde...
The C compiler pretends to be the computer. But UB is still there, as a compiler-only thing that has no representation at all on the computer.
C is like a fast motorcycle, rust is a car with driver-assistance system.
This is the #1 problem I see with people trying to learn a new language (not just Rust).
I’ve watched enough people try to adopt different languages at companies over the years that I now lean pessimistic by default about people adopting new languages. Many times it’s not that they can’t learn the new language, it’s that they really like doing things the way they learned in another language and they don’t want to give up those patterns.
Some people like working in a certain language and there’s nothing wrong with that. The problems come when they try to learn a new language without giving up the old ways.
Like you, I’m getting similar vibes from the article. The author wants to write Rust, but the entire premise of the article is about not wanting to learn the rules of Rust.
Definitely! I've also noticed people will learn a group of similar languages, like Java, C#/.Net, then Kotlin as the most distant relation. Now, they think they know many languages, but they mainly have the same core idea. So when they try something new like Haskell or Swift or Rust, they think it's doing something different from the "norm" in a really irritating way.
In Python, I frequently see the same problem from the other side. Instead of C/C++ programmers learning Rust and "not wanting to learn the rules of Rust", it's Java/C# programmers learning Python and not wanting to unlearn the rules of Java/C#. They write three times as much code as they need to - introducing full class hierarchies where a few duck-typed functions would do.
i had similar problem with tensorflow for like a year or so until it “clicked” i was trying to make the framework do thing the way i wanted, instead of doing things the way the framework wanted.
seeing that in myself was a real eye opener.
now i see it in others too and it’s frustrating because, ultimately, no-one can tell them they’re doing it. they have to find out for themselves for it to make a difference for how they act.
Obviously, the borrow checker has uses in preventing a certain class of bugs, but it also destroys development velocity. Sometimes it's a good tradeoff (safety-critical systems, embedded, backends, etc.) and sometimes it's a square peg in a round hole (startups and gamedev where fast iteration is essential)
This is not a bad thing by the way, it's an extremely plausible design chocie, and is one that Rust made very clearly: rejecting not-entirely-correct programs is more important than running the parts that do work. Languages that want to optimize for prototyping will make the opposite choice, and that's fine too.
Rust isn't alone in this, languages with type hints are currently going through the same thing where the type-checker can't express certain types of valid programs and have to be expanded.
This is fixable. Because you can have back references. You just have to use Rc, Rc::Weak, .upgrade(), RefCell, .borrow, and .borrow_mut(). This works, but only if the upgrades and borrows never fail. A failed .borrow() is a panic. The implication is that if you use .borrow() or .borrow_mut(), there's some good reason to think it will never fail.
For Rc::Weak, the key constraint is that all weak pointers must drop before all strong pointers have dropped. If you can prove that, .upgrade() doesn't need a run-time check.
For RefCell, the key constraint is that no .borrow() or .borrow_mut() may be enclosed by the scope of a conflicting .borrow() or .borrow_mut(). This requires a transitive closure check on who borrows what. For many simple cases, this is statically checkable. It does require inter-function checking.
Can those checks be moved to compile time? Probably. There's already a compile-time static Rc.[1] Compile-time RefCell checking looks possible.[2] It's non-trivial to do this, but worth thinking about.
DARPA's TRACTOR project (Translating All C To Rust) is likely to generate vast amounts of Rc-heavy code, if it works. So that provides some motivation for doing something to check at compile time.
[1] https://github.com/matthieu-m/static-rc
[2] https://internals.rust-lang.org/t/zero-cost-interior-mutabil...
Then you can have dangling pointers that point to the wrong place. Race conditions become possible. You need an allocator for the indices. I've had problems with a renderer that works that way. On rare occasions, it crashes with an invalid index. Even with Rust safe code.
Generational indexes in a SlotMap or similar should rule this out.
> Race conditions become possible.
How so?
> You need an allocator for the indices.
Yes, this usually becomes a central point of synchronization. (If you just have a SlotMap, you might not think of it as an "ID allocator", but that is half of what it is.) It doesn't have to be fully synchronous, though, and you can do fancy things with atomics.
> On rare occasions, it crashes with an invalid index.
Indexes become a sort of Weak pointer, and you get an Option when you dereference them, so you can certainly crash if you .unwrap() it or similar. You can use a Vec with "infallible" indexes if you don't want to support deletion, but if you do want to support deletion, the deleted case can bite you one way or another, no?
Right. I use a renderer where, on rare occasions, that happens. I didn't write it, but now I have to fix it, because it's abandonware.
This is completely back to front. Of course you have to internalise the rules of a borrow checker or type system to be highly productive. How can you hope to do a good job without that?
My take-away was that the article concludes that Rust is NOT a good tool for working with the borrow checker. I don’t think it said that there’s anything wrong with a borrow checker, and that you shouldn’t learn the basics of how it works or how to write idiomatic Rust.
A good tool shouldn’t require you to have a perfect memory of all the rules for you to be highly productive with it. If you make mistakes it should quickly tell you so with a message that quickly lets you figure out what to change.
I think this stands in contrast with Zig where these goals is the highest priority of the language. It’s also very strict with little to no undefined behaviour. But there’s also a lot of discipline in not introducing syntax or semantics that makes it hard for the compiler/checker to give a quick pass/fail with a clear message about went wrong. You can see from the issues in GitHub that improving error messages for failures in the type system is consistently prioritised. That puts a hard constraint on Zig where they’re held back from putting too much power into the type system.
That’s not to say that Rust doesn’t prioritise being a good tool. But the semantics of borrow checking makes their job an order of magnitude more difficult here. It’s an inherent trade-off. They’ve made a huge jump in the complexity and power of the language, and it’s probably much harder to then make a tool that makes it comfortable to work with this kind of power.
Some here may find it easy to deeply understand all the rules and to write code the first time that doesn’t trip up Rust too much. But in the real world code is written by many different kinds of people with different kinds and levels of intelligence.
I’ve found this to be an important consideration when choosing languages, libraries, tools and methodologies for large teams.
One way to be a 10x programmer is to write 10x as much code as an average programmer. Another way is to make 10 other average programmers twice as efficient, and that’s clearly more scalable.
Rust may still be a good choice to make the team more productive in the long run. My point is that adopting it in a team should perhaps not be considered a trivial decision.
> A good tool shouldn’t require you to have a perfect memory of all the rules for you to be highly productive with it. If you make mistakes it should quickly tell you so with a message that quickly lets you figure out what to change.
That's exactly what the Rust borrow checker does for me.
The borrow checker rules are quite simple conceptually.
If I own a book, I can read it, write in the margins or even destroy it. `let book: Book = Book{...}`.
I can lend this book to you exclusively `&mut Book`, you can read and write it, but not destroy it. And nobody else; including me; can even read it until you are done.
I can lend this book to you and others for reading `&Book`. We can all read it concurrently. And I must wait for everybody to be done before I can regain full ownership.
I can give you the book (passing by value). And it's now yours to do what you please. Including destroying it `drop(Book)`.
Sometimes you do want to share to many; and maybe even gate exclusive write access; at runtime. This is where Rc, Arc, Cell, RefCell and Mutex come in.
Rc and Arc destroy the book when a reference counter drops to zero. Another way to look at it, is that when the counter is 1, you have sole ownership of the book. And you can do with it what you please.
As for the runtime check for mutability, Mutex should be obvious. Cell and RefCell are similar but within a single threaded context.
And finally when you know better than the compiler, you use pointers (instead of references) and triple check your work within `unsafe` blocks.
This is generally a good thing: the more you internalise the logic of borrow checking, the earlier you start thinking about "who owns what" instead of deferring the choice to later, which often ends up in a tangled mess of "incidental data structures" as it is sometimes called in the c++ world [1].
Of course in c++ this means you have to internalise this discipline the hard way, i.e. without the borrow checker helping you.
[1] https://isocpp.org/blog/2016/05/cppcon-2015-better-code-data...
Maybe there's a style I haven't learned yet where you start out with Arc everywhere, or Rc, or Arc<Mutex<T>>, or whatever, and get everything working first then strip out as many wrappers as possible? It just feels wrong to go around hiding everything that's going on from the borrow checker and moving the errors to runtime. But maybe that's what you need to do to prototype things in Rust without a lot of pain? I don't know enough to know.
I have already noticed that building up the mindset of figuring out your ownership story, where your values are being moved to all the time, is addictive and contagious -- I'm sneaking more and more Rusty ways of doing things into my C++ code, and so far it feels beneficial.
I wouldn't recommend that. It's easy to end up with a fundamentally flawed architecture impossible to refactor out of.
In general as long as you stick to keeping data ownership as high up in the call stack as possible everything should slowly fall into place.
Think functional core imperative shell.
Your main has services, dependencies, data, and just makes calls that operate on data without trying to make deeper owned objects that are inherently hard to keep references to.
There's some truth to that, but I think the real problem is when some part of the state of your program looks like a graph with loops and cycles (not a tree). It's possible that it only looks that way during prototyping, but I think it's more likely that once it starts looking that way, it's gonna stay that way. In that case, "hiding" your lifetimes from the borrow checker is really about making your borrows shorter, which is how you can manage a graph without violating the no-mutable-aliasing rule.
I guess the hard bit here is it's more difficult to make a mental model of the borrow checker than it is most other features, and also the fact that borrow checking is a late phase of the compiler so you might put in a fair amount of work before getting feedback.
My experience doesn't match this. I think the difference is that Rust has... "specific types", to pick an arbitrary term. The types store more information, which means that there are many more ways that you might need to update an interface.
An interface with a function that takes a C++ pointer doesn't imply anything other than that the pointer is valid at the beginning of the calthe return value of l to that function. It doesn't even tell you whether it can be nullptr or not. It might be a pointer extracted from a `std::unique_ptr`, in which case the actual expectation is that the pointer be valid when the function returns. Or it might be passed the return value of `new`, in which case the expectation is that the pointer be either valid but now owned by something else, or invalid when the function returns. And you might change from one expectation to the other merely by removing a `delete` call 7 levels deep, without needing to adjust any of the intervening layers.
Rust, on the other hand, encodes a specific subset of the possible lifetimes that the C++ version accepts, and every intervening layer has to agree on that lifetime or at least the structure of it.
And it's not just lifetimes. In Rust, you'd typically make everything a specific type -- maybe an enum or Option or whatever. In C++, I often find myself intentionally degenerating even (C++) enums to ints when I need to store or output or manipulate them -- I want to test `val < FirstCustomValue` or something. A parameter that started out as an enum with 3 variants can add on a couple of orthogonal bits without perturbing much of anything.
The more specific the types used in practice, the more they'll need to be adjusted to accommodate changes. C++ doesn't even have the facilities for specializing the types as precisely, but more importantly types are typically left pretty loose in practice -- at great cost to stability and correctness, but with resulting benefit to adaptability.
A lump of mud is much more adaptable than a carved sculpture.
Rust happens to check quite a lot. I imagine it is similar for things like monads in Haskell, or probably formal languages like Lean and Dafny... but I have no experience of that so it's just a guess.
Wait a minute... :-)
Sorry, somehow I managed to quote the portion of your message that I was agreeing with. You just said it more concisely than I could.
The aspect I disagree with is whether brittleness to change is more due to the complexity of the borrow checker vs interfaces with very constrained types. I mean, the complexity of the borrow checker definitely factors into it, especially with how late it runs (as you said) and the fact that it's all or nothing, which makes it hard to change things incrementally.
But (imho) the borrow checker is less complex than, say, the rules around C++ constructor variants. And RVO/NRVO. And name lookup. And definitely less complex than undefined behavior. I'd rather memorize the borrow checker than those. They contribute to some amount of brittleness to change, especially when monkeying with move constructors, but in practice it seems far less than Rust. I really think it's more about the specificity of types in the language.
I should probably mention that I still think it's a good tradeoff for many of the types of programs I write, but it is a tradeoff.
Anyway, this is all more just a realization that you sparked in me. It's ok if we, uh, disagree to agree.
That said, the tedious refactors are a real pain. I think we all hoped that rustc would be smarter by now. It has gotten better but it isn’t there yet.
This only works for projects which do not involve any R&D, but have a complete and well written functional specification written in advance. Also for projects which do a complete re-implementation of some pre-existing software.
For greenfield projects which require substantial amount of R&D, it’s impossible to architect programs ahead of time. At the start of the development, people only have a wishlist. Architecture comes later, after several prototypes implemented and evaluated, and people have some general understanding what does and doesn’t work, and what specifically needs to be done.
Rust implies that upfront architecture costs even for prototypes.
Just because an algorithm is not representable in safe rust doesn’t mean it’s a bad idea. See my other comment in this topic https://news.ycombinator.com/item?id=42164582 You’ll find similar parallel algorithms in all high-performance BLAS libraries. On my day job I sometimes do similar things in C++, using OpenMP or other thread pool for parallelism.
I don’t like that balance.
For performance-critical things like BLAS and other low-level numerical stuff, C++ is safer than unsafe Rust because standard library and tooling (debug heap, debug allocators, ASAN, etc.) evolved for decades making the language usable despite the unsafety. Another thing, for numerical code you probably need SIMD intrinsics. They were defined by Intel, documentation and other learning resources almost exclusively target C language; C++ got them for free due to compatibility.
For high-level pieces which are not that performance critical, I use C#. Due to VM, it’s safer than Rust. Due to GC and other design choices usability is way better than Rust, e.g. in C# asynchronous I/O is almost trivial with async/await. The runtime is cross-platform, I have shipped production-quality embedded ARM Linux applications built mostly with C#. Unlike Java, it’s easy to consume unmanaged C++ DLLs using C API, or even C++ API: see that library to do that on Linux which doesn’t have COM interop in the runtime https://github.com/Const-me/ComLightInterop
I don't memorise how it works, I've just learnt what it rejects and why, and this in turn becomes clear as to why it's rejected that. Very rarely do I find myself going "oh bother, now I suddenly need to `Rc` or `Arc` this, I suspect because I've just gotten into the habit of suspecting when I anticipate things will run afoul and structuring things from the get-go to avoid that. Admittedly, I'm not writing absurdly low-level code.
I wonder if the authors grounding C++ is making life harder for them? Often when I've had to teach people Rust, getting them to stop writing {C/C#/Java}-but-in-Rust is the first stop on the trail to "stop fighting and actually enjoy the language". Every language has its idioms, just because you can, doesn't mean you should.
I used to think Arc/Rc was a shortcut to avoiding the borrow checker shenanigans, but have evolved that thinking over time.
You do mention it in your comment so wondering if you have anything to share about it
I’ve had more than enough unpleasant experiences with write-runs-breaks-at-runtime style languages. I don’t like writing them, I hate having to support them in prod as they give me constant-persistent-stress. I hate what idiomatic C# is, and how much ceremony there is to read and write it. Java is worse. Go’s type system is too anaemic for my tastes. Haskell is nice, but gets a bit academic and lacks some day-to-day niceties. Kotlin is supposed to be nice, but again, we’re getting maybe 50% of Rust type system features, and you’re basically just piggybacking on Java/JVM which I hated dealing with previously. IDK what else that leaves in the mainstream. I used to play around with Nim, and that was quite nice though.
Expression-oriented, HM type inference with gradual typing, faster than other FP languages, can even reach for low-level bits, or write extra glue code in C# which is more pleasant at low-level imperative code.
Not sure what exactly you refer to as idiomatic C#. If it has too much ceremony chances are it’s anything but!
Edit: I do love the ML style syntax though, Haskell, F#, Dhall are awesome, I wish it were more readily accepted.
I recognise C# as useful, but _personally _ dislike it because of the sprawling, OOP-heavy, ceremony-heavy core it encourages you to write. Half my dev friends write C# for their day jobs, half my workplaces have had me dealing with it. It’s not an opinion borne from unfamiliarity or knee jerk reaction; my distaste for MS is just the cherry on top.
> This is painful because I am an experienced C++ programmer, and C++ has this exact problem except worse: undefined behavior. In the worst case, C++ simply doesn’t check anything, compiles your code wrong, and then does inexplicable and impossible things at runtime for no discernable reason (or it just deletes your entire function).
This is completely wrong, even in the "not even wrong" territory. It reads like an attempt to parrot a cliche without having any idea what it means. "Undefined behavior" just means the standard does not define what is the expected behavior, and purposely leaves implementations free to implement it how they see fit. This means crashing the app or sending an email to the pope.
In practical terms this means developers should not write code that triggers undefined behavior, and treat the code that does as errors requiring a fix. Advanced users can lean on implementation-defined behavior from compilers to add some expectation to the behavior, but that's discouraged.
It's so strange how someone calling themselves a seasoned C++ developer fails to understand such a basic aspect of the language.
The important tidbit is that a) it's completely wrong to parrot "undefined behavior" on "C++ doesn't check anything", and b) if you code triggers undefined behavior without your knowledge then you just broke the code and wrote a bug out of your own ignorance.
To make matters worse, there are a myriad of code checkers for C++ that catch undefined behavior and even some classes of safety errors. Take for instance cppcheck. Why is the blogger whining about undefined behavior and "c++ not checking" when adding cppcheck to any project is enough to detect most if not all cases?
> This means that in order to write C++, you effectively have to memorize the undefined behavior rules, which sucks. Sound familiar?
Which is the point TFA is making. I believe you expended the attention span a bit too early.
Again, the point is wrong in more than one way.
You don't simply add undefined behavior. It's wrong, and buggy code. Onboard a static code analysis tool like cppcheck to tell you when you messed up.
It takes far less work to onboard any of these tools than it takes to write a sentence in a blog post.
One of the other things that makes me worry about Rust is how similar it's depends look to npm projects, where there's a kitchen sink of third party (not the language's included library of code, and not the project's code) libraries pulled in for seemingly small utilities.
1. End up with a third party package filling in the gaps, or
2. Another standard library API that users slowly migrate to
We'll see how this works, Rust is still young and not yet used in any hugely important projects (or at least not in hugely important parts of those projects - e.g. some Linux drivers, not the core kernel; some bits of Firefox'S rendering, not the JS engine). As it becomes more central, it's value as an attack target will increase, and people will start taking infiltrating malicious code in small but widely used dependencies.
Having a large and good standard library, supplied by a single trustworthy foundation, with dedicated employees that check incoming PRs, is going to become more and more important in the following years.
Trying to anticipate all or even most use cases in the standard library is a fool's errand (unless we're talking about a DSL, of course). There are too many and they are too dynamic to be captured in the necessarily conservative release process of a language implementation. Languages should focus on being powerful and flexible enough to be adapted to a wide variety of use cases, and let the community of package maintainers handle the implementation. Think of this as a special case of the Unix philosophy; languages should do one thing very well, not a million things unevenly.
I bet most people here don't believe a command economy could ever work in a market for goods and services. Why should it work in a marketplace of ideas?
If the suggestion is that putting things in the standard library makes them better, I disagree. My experience with Python for instance is that a "batteries included" strategy results in some phenomenal packages and some borderline abandoned packages that are actively dangerous to use.
To riff on your metaphor, the federal government designs the arterial highways, but the state, country, and city/town officials design the minutia of the traffic system. If the federal government had to approve spending on replacing some street signs or plowing snow, we would have a terribly impoverished transportation system.
For two examples: plenty of languages leave auto-formatting and testing to the community, functionality which rust is better for having standardized.
I usually like to evolve a code base towards a new architecture a little at a time, keeping it running and passing tests at every step of the way. What I found was that even seemingly small changes required an awful lot of work, as the OP says; if I could make them work at all. Eventually I decided that I’d learned what I’d needed to, and walked away from it. (To be fair, this was late spring or early summer of 2020, everything was peculiar, and I didn’t have the spare mental capacity for the project.)
I should add: I understand the need to use a language the way it wants to be used, and that you need to assimilate and internalize that to be truly fluent. I concluded that I didn’t need Rust’s extreme performance for the kind of work I do, and that there are less intrusive ways of getting memory safety.
True that.
On one hand, it's amazing. On the other hand, the nagging feeling that we still have work to go in programming language design has not gone away.
Structured concurrency. Don't provide "legacy" mechanisms and "opt in" for structure, just bite the bullet. Like the first language which told people no, we don't "go-to" other functions, that's not happening in my language, that was structured program flow I want structured concurrency, it's a thing but it's not yet popular enough to do that as the only provided concurrency, it should be.
Smart arithmetic. Your computer has Floating Point math. FP math is fast, but, it's hard for humans to think about exactly where they lose precision and performance while using it. I should be able to write the real mathematics I want, specify the precision I need and possibly the performance trades I'm comfortable with, and the compiler not me the programmer, figures out how to use FP math to calculate my mathematics with acceptable precision or tells me that I made demands it couldn't meet or which are nonsensical.
On the other hand, two things I really liked to see when I learned Rust:
&[T]: The slice reference type, a fat pointer which specifies where zero or more contiguous Ts are, and, how many of them. This is the Right Thing™ and it's right there in the core language design, which means you don't need to go back and retro-fit it.
String: The simplest possible way to build the growable text type, as a growable array of bytes but with the strict requirement that the array's content is always UTF-8 text. Is the "Small String Optimization" a cool trick? Yeah, but it need not live in this core vocabulary type. How about Copy-on-write ? Ditto. What about other text encodings? Transcode at the edges if you need that.
So it's like discussing politics or religion. People think that they have objective views, but they can't overcome their beliefs. That's just how beliefs work. They almost never change.
Also, beliefs are tied to groups. Humans automatically adopt the beliefs of their group, at least to some degree. Or they learn to shut up about their disagreements.
This is a thread for Rust critics and Rust advocates. Try to seriously sell F# or some other ML-like language in here and you are going to end up annoying both the C++ people and the Rust people.
The world will be a better place when the AIs finally take over. If we survive.
What does that say about participating here? Well, for me, sometimes when I write a comment that I feel is constructive, reasonable, and honest, it goes gray anyways, and it's easy to chalk it up to people just irrationally downvoting it because they don't like my opinion. It's also pretty easy to do this, I just need to be cynical about Apple or optimistic about the Go programming language, or something similar, and there's some percentage chance it will go negative depending on presumably who sees it first. It's not going to stop me from doing so, and ultimately it's pretty inconsequential, as I'm just some guy and my opinions are not really that important anyways.
Somehow, even though I have all of this internalized, I can't help but go 30 nested replies deep into threads debating about something senseless and unimportant, but it almost feels like it wouldn't be the Internet without debates like that. XKCD 386.
However, managing ownership and lifetime _is_ managing memory. The borrow checker is there, all the time, reminding you of memory management.
Now, in C and C++ the same problem exists but you don't have a borrow checker to remind you. I think this is the same conclusion the blog post came to, but I'm not entirely sure.
What are people writing that requires such fancy/extensive usage of the borrow checker?
I can't even remember the last time I had to use a shared_ptr... unique_ptr and other general RAII practices have been more than enough for our codebase.
Who is Rust for?
Rust is for anyone who finds it useful.
Me too.
> What are people writing that requires such fancy/extensive usage of the borrow checker?
The simplest example I can imagine is this: https://en.wikipedia.org/wiki/Matrix_multiplication When your matrices are large and you want it to run fast, you want to parallelize.
Good algorithms (which don’t bottleneck on memory bandwidth) need multiple CPU cores to concurrently store different elements of the same output matrix. Moreover, the elements computed by different cores are not continuous slices, they are rectangular blocks. Such algorithm is not representable in safe rust.
Having said this, the benefits of borrow checker out weight the shortcomings. I can feel myself writing better code in other languages (I tend to think about the layout and the mutability and lifetimes upfront more now)
My rust code now is very functional, which seems to work best with lifetimes.
I would love to know more about the authors pain, I do hope rustc gets better at lifetime compilation errors cause some of them can be very very gnarly.
When this happens, file tickets! We do our best to improve diagnostics over time, but the best improvements have been reactive, by fixing a case that we never encountered but our users did.
I know my way around this now, which is to literally binary search over the timeline of my edits (commenting out code and then reintroducing it) to see what causes the compiler to trip over (there might be better ways to debug this, and I am all ears)
Most of the times this error is several layers deep in my application so even tho I want to ticket it up, not being able to create a minimal repo for anyone to iterate against feels like a bit of wasted energy on all sides, do let me know if I should change this way of thinking and I can promise myself to start being more proactive.
Even if you have already figured out how to deal with it, your future colleagues might not, and by improving the diagnostic you would also be getting that time manually commenting code back.
In the same vein: «C/C++ can be hard to get right because of the valgrind.» ;-)
If I'm working on a section of code the relies heavily on borrowing and lifetimes, I will typically work up a prototype without all the functionality just to ensure I have a workable design before going back to fill in the rest of the code. This is probably why I don't tend to hit it all that often. It would be ideal if this wasn't necessary, but Rust has all sorts of other awesome features that make this something worth enduring.
We are always on the lookout for improving in this area. Having examples of cases where we conceivably should have done better but didn't is useful. As mentioned already, the complexity here is that doing the right thing for the user requires architecting multiple separate stages of the compiler to talk to each other in way more complex ways than originally intended.
> Unfortunately, it can only catch undefined behavior that actually happens, so if your test suite doesn’t cover all your code branches you might have undefined behavior lurking in the code somewhere.
Covering every branch is not enough to say you have full coverage.
if( condition1 ) {
/* complicated things */
}
if( condition2 ) {
/* complicated things */
}
if( condition3 ) {
/* complicated things */
}
There are a few genuine cases which the BC won't accept, though they may be valid. The first case is of data structures containing cycles (like dequeues, ring buffers, closed graphs, etc). The other is cross-FFI calls. This is due to the fact that the BC simply doesn't have the intelligence to reason about them (at compile-time). Even then, Rust gives you 2 types of escape hatches - runtime safety checks (using Rc, Cell, etc) with a slight performance impact, and manual safety checks (unsafe) when performance is paramount. All that's expected of you (the programmer) is to recognize such cases and use the appropriate solution.
I'm not too surprised when non-C/C++ programmers struggle with BC rules. They may be unfamiliar with the low-level execution semantics. But I'm surprised when C/C++ programmers fail to make the connection. I was a C++ programmer too and this is the first thing I noticed. Memorizing the BC rules is the absolute worst way to learn it. You should be looking for memory safety problems and correlating them with BC error messages instead. I know this works because I trained non-systems (non-C/C++, primarily JS and Python) developers in Rust. They picked up the execution and memory semantics quickly and easily made sense of the borrow checker idiosyncrasies.
What this means in practice is that Mojo's lifetime checker extends the life of values. Just point it at an origin and it'll ensure the origin is still alive wherever you use the value attached to it.
It completely defines away "value does not live long enough" compiler problems.
Like it's not that it never caused issues but most times fixing them also produces much better code.
In my experience the most common place to run into issues is if you write C/C++ style code in rust.
Or if you write certain kinds of functional style code in rust, rust has many functional features but isn't strictly speaking a functional language and while some functional pattern work well many other fall apart especially if combined with async (which will get better once async closure are stabilize).
in the end it often boils down to trying to use patterns and styles for other languages in a language which doesn't support them well, which always causes issues, but most times (in other languages) in more subtle ways then compiler errors, e.g. UB, perf, etc.
Through there is one field (game programming) where as long as your project doesn't become quite big you can get away with a lot of suboptimal approaches of state handling but not in rust. So if it comes to hobby from scratch state game programming I wouldn't be surprised for people to get annoyed (but if it's game programming using existing frameworks and e.g. stuff like a entire component library like bavy it's a different topic altogether)
We have solved memory management; you can connect your objects every which way and garbage collection can deal with it.
We should use that for 99.9..% of the code that's out there. The rest could as well be written in assembler, except that we have multiple instruction sets to target.
There are some gotchas that you need to learn (e.g. self-referential structs won't work, or & returned from a &mut method won't be shareable).
But besides a few exceptions, it's mostly shared XOR mutable data in the shape of a tree. It's possible to build intuition around it.
My first preference for making simple utilities is as a shell script. The immediately next one is actually Rust.
The only explanation I can think of for the dislike towards Rust's compile time checks is that some people don't entirely understand these rules when they use C and C++. It's possible to resolve simple memory safety issues in C/C++ without in-depth knowledge of hardware semantics. But a complicate bug will easily stump you at runtime (personal experience).
In a broader sense, I keep seeing some people asking everyone else to avoid Rust based on an exaggerated account of the struggle with the borrow checker. There is actually a way to get comfortable with the BC. Perhaps beginners should be introduced to those ideas rather than such negative takes.
Because there are other languages which don't require you to hold to a strict set of rules to maintain memory safety, as they automate memory reclamation at runtime.
> There is actually a way to get comfortable with the BC. Perhaps beginners should be introduced to those ideas rather than such negative takes.
Regardless of how comfortable you are with it, it imposes a necessary overhead, either in terms of architectural constraints or the runtime cost of reference counting which is greater than that of garbage collection.
> I keep seeing some people asking everyone else to avoid Rust based on an exaggerated account of the struggle with the borrow checker.
The borrow checker is irrelevant here. As you have mentioned, a person coding is C will have to keep to the same constraints which Rust statically enforces. There is clear reason to use a language such as Java over C. Not having to manually manage memory simplifies code and makes refactoring easier. This is true regardless of if your memory restrictions are checked or unchecked. Imagine the simplest example: a data structure containing a string. In Java, you write a class and put a string in it. The data for the string is shared and a simple copy of the pointer will suffice in the constructor. In languages with manual memory management, you have three options: the struct can own the string (or reference count it), in which case it will have to be cloned and performance will be worse than garbage collection; the struct can borrow the string, in which case you'll have to track the string's lifetime throughout the program which could cause issues later; the structure can be generic over owned and borrowed string types, in which case code will be more complicated and generic parameters will proliferate. It should be immediately apparent that the latter requires more architectural consideration than the former, and hence is a greater burden to programmers. Several of the options in the Rust case will also lead to potentially complicated refactoring later, which adds even further cost to development.
TL;DR managing memory manually takes more effort. That's why it's called MANUAL memory management, because it's not automated.
Rust becomes a lot simpler when you borrow less and clone more. Sprinkle in smart pointers when appropriate. And the resulting program is probably still going to have fantastic performance - many developers err by spending weeks of developer time trying to shave off a few microseconds of runtime.
But, if you're a developer for whom those microseconds really do matter a lot, well then you just have to bite the bullet.
Anecdotally, all of the happy productive Rust programmers I know do not come from a hardcore systems background. They were mostly Java and Python developers that wanted to get a bit closer to the metal. For them it is probably a great experience, and the performance is an improvement relative to what they are accustomed to.
There are quite a few shops that use Rust and C++ together, often wrapping a C++ core (for performance) with a Rust API layer (for safety).
If you are used to C++11 or newer, you should be able to continue writing very similar code in Rust. The only major issue I encountered was the lack of the idea that "because objects A and B have effectively the same lifetime, they can safely store references to each other, as long as..." But if you are used to older versions of C++, trying to write similar code in Rust is going to be painful.
> This means, to be a highly productive Rust programmer, you basically have to memorize the borrow checker rules, so you get it right the first time. This is stupid, because the whole point of having a type system or a borrow checker is to tell you when you get it wrong
I'm not sure how you want to square this circle, you don't want to memorize the rules of UB, but you also don't want for compiler to correct you when you make UB behavior according to Borrow Checker?
The best way in both C++ and Rust is to structure your tree of lifetimes and use other means to achieve your desired goal.
But I have a hard time to envision any other solution than "better tooling for refactoring".
So I’d say it’s “worse” than “you have to memorize the borrow checker”. Its “you have to learn how to write programs in Rust”.
Know what tool to pick.
[1] https://doc.rust-lang.org/std/sync/struct.Arc.html#impl-Dere...
[2] https://rust-lang.github.io/rfcs/2005-match-ergonomics.html
match Arc::new("hello") {
"hi" => {}
"hello" => {}
_ => {}
}
match *Arc::new("hello").as_ref() {
"hi" => {}
"hello" => {}
_ => {}
struct Foo {
bar: Arc<String>,
}
if let Foo { bar } = val {
if &*bar == "hi" {
}
}
if let Foo { bar: "hi" } = val {}
And in any case, ecosystem usually trumps other concerns. Deep learning is a task you'd think a system language would be ideal for. Yet Python is a probably a better choice than Rust. The best reason to avoid Rust for a new project is probably that the market for Rust developers is insufficiently liquid (though presumably that won't be true for much longer, if it's still true at all).
Meanwhile, Rust is my programming language and I choose it unless there is a good reason to choose another language. I never really struggled with the borrow checker. I think a lot of beginners approach the BC wrongly. Trying to memorize the rule is definitely the wrong way.
Of course, that's always the trade off with Rust. You're trading a _lot_ of time spent up-front for time saved in increments down the road.
As a concrete example, I'm in the process of building up a Rust database to replace the Postgres solution one of my applications is using. Partly because I'm a psycho, and partly because I've gotten query times down from 20 seconds on Postgres to 50ms with Rust (despite my best efforts to optimize Postgres).
Being a mostly ACID, async database, this involves some rather unpleasant interactions with the Rust borrow checker. I've had to refactor a significant portion of the code probably five times by now. The lack of feedback during the process is a huge pain point, as the article points out (though I'm not sure what the solution to that would be). Even if you _think_ you know the rules, you probably don't, and you're not going to find out until 2 hours later.
The second most painful point has to be the 'static lifetime, which comes up a lot when dealing with threading and async. For me it's when I need to use spawn_blocking inside an async function. Of course, the compiler has no way of knowing _when_ or _if_ spawn_blocking will finish, so it needs any borrows to be 'static. But in practice that means having to write all kinds of awkward workarounds in what should otherwise be simple code. I certainly understand the _why_, and I'm sure in X years it'll be fixed, but right now ... g'damn.
That said, the borrow checker _has_ improved. I think my last major Rust project was before the upgraded borrow checker, which wasn't able to infer lifetimes for local variables. So you had to throw a lot of stuff inside separate blocks. We also have a lot more elided lifetimes now. Just empirically from this project I'd say me and the borrow checker only had about 30% of the fisticuffs we did in the past.
Personally, I think the tradeoff is worth it. It won't be for everyone, or every project. But 20s to 50ms query time, with a ton of safety guarantees to ensure the valuable data running through the database is well cared for? Worth every line of refactored code.
Asides:
* The project in question: https://github.com/fpgaminer/tagstormdb
* I also replaced some of my large JSON responses with FlatBuffers. FlatBuffers is a bit of a PITA, but when you're trying to shuffle 4 million integers over to the webapp, being able to do almost 0 decoding on the browser side, and get them as a Uint32Array directly is gold.
* It's a miracle I got away with the search parser in the project. I use Pest, and both the tree it spits out and the AST I build from it hold references. Yet sprinkling a little 'a on the impl's and struct's did the trick.
* Dynamic dispatch has also improved, as far as I can tell, which used to always involve some weird lifetimes if the return values needed to borrow stuff.
* ChatGPT o1 is a lot better at Rust then 4 or 4o. I've gotten a lot more useful stuff out of o1 this time around, including less hallucinations. Still weaker than Python/TypeScript/etc, with maybe 2-3 compile errors that need to be fixed each time. But still better. Sonnet completely failed every time I tried it :/ (Both through Copilot and the web). o1 in Copilot _could_ be amazing, since I can directly attach all my code. But the o1 in Copilot _feels_ weaker. I'm fairly sure the 4o Copilot uses is finetuned, and possibly smaller, so it too always felt weaker. Seems like o1 is the same deal. Still really useful for the Typescript side of things, but for Rust I had to farm out to the web and just copy paste the files each time.
Or at least I thought I did, until I launched into a project that mixes async and threading. That's where I hit a wall. What it's complaining about makes sense to me, but how to fix it does not -- partly because the async and threading come from libraries that I'm trying to stitch together. They necessarily have their own idiosyncratic ways of dealing with the issues, and as a beginner I don't even fully understand the problem they're solving let alone their solutions.
(For the record, I'm basically trying to force an unholy matrimony of matrix-sdk + tokio + pyo3 + pyo3-asyncio. You can take Python's GIL to grab out values to work with, but then if you want to do an async function then you'll have to release the GIL to stuff things into a future, and... well, if I fully understood the problem then I wouldn't be here whining about it, would I?)
Sweet summer child. Use Haskell for a while and get back to me.
1. Give priority to the fundamental semantics of code execution - things like C memory layout [1], function calls, stack, stack frames, frame invalidation, heap, static data, multi-threading/programming, locks, synchronization, etc. Also learn associated problems like double-free, use-after-free, invalidated references, data races, etc. These are easiest to understand if you learn an assembly language. However, if you don't have the time or patience for that, at least focus hard on the hardware and memory layout topics in Rust books. If you prefer instead to learn those by making mistakes without the borrow checker intervening, start with C. (Aside: This knowledge is needed for C and C++ as well, especially C. You can't write large code without it.)
2. DON'T try to memorize the borrow checker. Borrow checker is based on a few simple principles (which you should know), but they can manifest in very complex and surprising ways (same as memory safety bugs). It's not practical to learn all such cases. Instead, check the borrow checker error message and see if you can correlate it to any memory safety problems I mentioned above. While the borrow checker can seem very complicated and arbitrary, it's designed solely to prevent those memory safety bugs. Pretty soon, you'll be comfortable with correlating BC errors to such bugs, without having to worry about how the BC found them. Knowing the real problem will also make it easy to satisfy the BC and avoid fighting with it.
3. Understand the memory layout of data structure that you use. Ref: [2]. Borrow checker errors often require this knowledge to make sense. The same becomes crucial during debugging if you make such mistakes in C/C++. The BC wont even allow you to compile it if you make similar mistakes in Rust. You need it just to get the program to run.
4. Borrow checker wont solve every problem for you. It doesn't have the intelligence to reason it all. There are a few notable cases:
- Data structures with cycles (like closed graphs, dequeues, etc) and algorithms that deal with them. (BC prefers data structures in a tree hierarchy)
- Function calls across an FFI boundary (since the BC can't check that code)
- Valid cases according to borrow checker rules, but rejected anyway due to complicated lifetime analysis. These may eventually get resolved in a later Rust version. But such cases exist.
5. Most of the BC errors can be solved by simple code refactors. But in cases like above, you need to identify them (as a limitation of the BC) and look for an alternative solution. BC is a compile-time safety checker. The alternatives are:
- Runtime safety checks using concepts like Rc, Arc, Cell, RefCell, etc. They will pass the BC checks at compile time. But if memory safety is violated at runtime, it will simply panic and crash. It may also have a slight performance hit due to runtime checks. But this is most often very negligible, given the fact that most other languages are based entirely on such checks (GC, ARC, etc). You don't need to be too shy in resorting to these to get around BC complaints. Many Rust programmers do.
- Manual safety checks using unsafe. If the performance is absolutely important for you, you can use unsafe functions and blocks. Unsafe keyword activates some potentially memory-unsafe language features (like raw pointer de-referencing) that the BC doesn't vet (Note: BC is not deactivated). This is often what you need when you're trying to implement an unavailable data structure or algorithm. This is also the only choice for FFI calls. Rust will not check them at any time. But this usually isn't a problem. Unsafe blocks are often used only for very fundamental ideas (eg: self-referential structs) and consist of no more than 5% of a codebase. If a memory safety bug does occur, it will be in one of those blocks and will be easier to locate and correct. Moreover people convert them to libraries and publish them on crates.io. This improves the chance of finding any hidden bugs. If you need unsafe code, there's probably a library for it already.
To get a better intro, check out 'Learn Rust With Entirely Too Many Linked Lists' [3]
[1] https://www.scaler.com/topics/c/memory-layout-in-c/
i’d consider myself a day-to-day c++ engineer. well, because i am. i like lots of things from rust. there’s a few things i don’t. c++ has a lot to learn from rust, if it is to continue to exist.
but really.. isn’t this the point of the language? you need to understand the borrow checker because.. that’s why it’s here?
maybe i’m missing something.
The overall point is it’s impossible to make a perfect spec. Rust doesn’t even have a spec, so your behavior is whatever the compiler gives you, which is C++ UB too.
The language specification, or standard, guarantees certain things about the behavior of programs written to the specification. "Undefined behavior" means "you did something such that the guarantees of this specification no longer apply to your program." It's pretty much a worst case scenario in terms of writing programs. The program might do... anything. Fortunately, in reality it happens all the time and programs often keep behaving close enough to what we expect.
Turing completeness is unrelated to that sense of "undefined behavior".
> halting
They are not entirely unrelated. C++ UB is often things that would be very difficult to detect.
For example infinite template recursion is undefined. Specifying any other behavior is impossible due to halting problem.
Another example: a system might be able to detect out of bounds pointer deref, or maybe not. Same with signed integer overflow.
Well, your point is wrong because UB is not an inherent computing problem. That's what the post above tried to explain.
Many forms of UB are inherent to C-like languages, but languages don't have to be C-like.
> For example infinite template recursion is undefined. Specifying any other behavior is impossible due to halting problem.
A language can avoid this by not having infinite template recursion.
C++ currently allows infinite recursion at the language level, while acknowledging that compilers might abort early and recommending that 'early' is a depth of 1024 or higher. But a future version could bake that limit into the language itself, removing the problem.
> Another example: a system might be able to detect out of bounds pointer deref, or maybe not. Same with signed integer overflow.
A language can avoid out of bounds deref in many ways, one of which is not allowing pointer arithmetic.
Signed integer overflow is trivial to handle. I'm not sure what problem you're even suggesting here that the person in charge of the language spec can't overcome. C++'s lack of desire to remove that form of UB is not because it would be difficult.
How does it know whether a definition is infinitely recursive? This IS the halting problem.
> But a future version could bake that limit into the language itself,
In other words, take away the Turing completeness of templates. Which goes back to my original comment.
Also note that limiting recursion hurts real world use case (types getting arbitrarily complex in a program over time) in favor of theoretical benefit (now you can say it’s not UB).
> pointer Deref
Let me explain again. In a language with pointers checking whether a deref is valid requires comparing every address to every allocation bounds. That’s ridiculously expensive.
The only solution is to take away pointers (Java, C#, etc) OR do what C does, crash on obviously bad derefs. Since “obviously bad” depends on the implementation (maybe you are a safety sadist and you want 2000 instructions per deref) the standard cannot guarantee any behavior. Maybe it crashes, maybe you get “lucky” and it doesn’t notice.
The only way to avoid UB is to limit expressiveness (pretend addresses don’t exist), and all Turing complete languages have UB.
I have more responses, but you’re not grasping the ones I already made.
Template halting is only a correctness issue because it's done at compile time. Turing completeness is not a problem in general, and limiting the amount of computation at compile time is fine.
> The only solution is to take away pointers (Java, C#, etc) OR do what C does
Something along those lines.
> The only way to avoid UB is to limit expressiveness (pretend addresses don’t exist), and all Turing complete languages have UB.
The only way to avoid UB is to limit expressiveness, and NOT all Turing complete languages have UB.
Pointers are a big thing to restrict for a safe language. But you really don't have to do that much else. Whether a program halts or doesn't halt at runtime isn't a safety issue, there's no UB involved. It just runs indefinitely.