Ask HN: How do I understand Rust? | Hacker News Reader

Ask HN: How do I understand Rust?

Serious question. I've been trying to learn Rust for a month now and I keep end up fighting the compiler with borrowing errors.

I'm really not sure where to turn for help. It seems like everything I know about structuring a program goes out the window when borrowing comes into the picture.

Thanks.

View on HN

> It seems like everything I know about structuring a program goes out the window when borrowing comes into the picture.

Try to avoid structured programming and mutable state where possible. Apply functional programming idioms and use immutable data structures if you can. Rust adopts many features from functional programming languages: http://science.raphael.poss.name/rust-for-functional-program...

Borrow checker will interfere only when you are trying to manage mutable state that is hard to reason about. If you absolutely must keep some mutable data structure around, you can fall back to unsafe code. There is nothing wrong with it, but make sure to put all unsafe code into separate module and provide only safe interface, that is absolutely unbreakable no matter how you use its public API.

Make sure you know about this module: https://doc.rust-lang.org/std/mem/ It is nearly impossible to create complex data structures without it, even linked list implementation requires it. Learn to use replace, swap and .take(). These functions provide great example of a safe API for unsafe operations.

> there's nothing wrong with [unsafe code]

I suppose I must ask, then: why Rust?

The point is to contain them and hide them under nice safe abstractions. If you avoid unsafe rust at all costs, including unsafe rust hidden under abstractions, you'll find that you can't write any rust at all because the standard library is so full of unsafe rust.

If you keep unsafe code in modules and don't pass raw pointers in between, it is much easier to audit the code. All the glue code is automatically safe. Then each module can be audited separately to make sure it is impossible to make it crash by using public API.

Rust `collections' library is full of unsafe code. Just for example: https://doc.rust-lang.org/src/collections/up/src/libcollecti... But you can never leave a BTree in an invalid state by adding or removing elements. Given safe collections library you can write lots of applications without resorting to unsafe code and having to prove the safety of your code.

Proving safety of B-Trees, threaded code and things like that requires a whole theorem prover that Rust most likely will never provide. You need to use Coq or similar tool if you are developing new algorithms and data structures. But if you are not a pure computer science researcher, most likely you don't need it. All you have to do is to carefully translate pseudocode from paper into Rust and package it.

I have to second the use of immutable data structures. That is the style I'd adopted prior to using rust, and in learning Rust I've found that I've hardly ever fought with the borrow checker.

Suggesting unsafe code seems like a bad move, if haven't mastered the borrow checker, is learning the dark magic of unsafe a good plan?

> functional programming idioms

Proper tail recursion was still missing last time I checked. Does it affect functional programming in Rust?

It's rare to actually use tail-recursive algorithms in Rust since those can be written with a loop, which Rust does support.

I mean, it's rare to actually manually recurse in Haskell too, and when you need to, it's often not something you can write tail-recursively.

>it's rare to actually manually recurse in Haskell

I wonder if there is a semi-easy way to measure this. My impression was that while the papers and tutorials are all about combinators, the people writing Haskell "in the wild" tend to go with general recursion more often than you might expect.

Even if you recurse, it doesn't mean you can make it tail-recursive. It might be a style issue, but Rust programmers are just going to use combinators instead anyway since recursion is slower.

It's not guaranteed, though LLVM may do it if it feels like it. We may gain guaranteed TCO in the future, we'll see.

The rest of the points here are excellent, but don't use `unsafe` until you have completely internalised the borrow checker rules. If your safe attempt is unsafe (and therefore doesn't pass the borrow checker), your unsafe code will be unsafe too. The only difference is that the unsafe code won't be caught at compile-time.

This is not true, in general. The borrow checker has many limitations where it cannot prove that something is safe, so it assumes it is unsafe. The classic example is borrowing disjoint slices from an array.

The borrow checker has limitations but I agree with gp: if you haven't fully internalized its rules, especially if don't understand why it is yelling at you at some point, then you should not go unsafe because you're most likely going to write memory bugs.

> Apply functional programming idioms and use immutable data structures if you can.

How would you implement efficient and flexible graph algorithms using these techniques? E.g., considering that edges need to be traversed quickly, updated quickly, etc., and graphs are arbitrarily complicated (not just trees or DAGs).

Curious: what kind of graph structures do you typically find yourself writing? Maybe I'm just doing different stuff, but I find myself with tree or DAGs in almost all cases.

Indirect the graph connections. This is almost always a better design anyway. Instead of storing pointers from one graph node to another, give each node an identifier and store a map from identifiers to nodes. Each node then has a set (or other collection) of identifiers it's connected to.

You can use either a mutable or immutable map for this construction.

That's basically the "don't keep references into a Vec, keep indices" advise. I don't really like that because references explicitly encode the dependency in the type system. When you start juggling indices you're basically implementing poor man's malloc. You need to make sure you don't lose track of any of them.

I think owned objects should have lifetimes and a "points to non-moving memory" property. E.g. a vec can be moved and must outlive all the objects that take references from it. That way it would be possible to keep an owned object and refs to it in the same struct and pass the whole bundle around.

> You need to make sure you don't lose track of any of them.

This is a fair point, but note that in any memory management scheme except garbage collected, you run the risk of having a dangling reference if you use direct pointers to other nodes. At least with node identifiers, you can safely check that the other node still exists, in any memory management scheme.

Store your graph as an adjacency matrix, using either a 2D array of edges (if your problem is small enough), or as a sparse matrix (hash table or similar).

There's a decent amount of research into functional graph algorithms. The right approach depends on the application, but two interesting ones are inductive graphs[1] and zippers[2].

I also agree with other commenters that you should make certain you actually need your arbitrarily complicated graph, though.

[1]: http://web.engr.oregonstate.edu/~erwig/papers/InductiveGraph... [2]: https://www.cs.tufts.edu/~nr/pubs/zipcfg.pdf

> Apply functional programming idioms and use immutable data structures if you can.

This is what I've tried to do when learning Rust. However, I find it ends up more boilerplatey than a language like Haskell. I had to do lots of clone() and iter(), which cluttered up the functional idioms. Maybe I'm doing it wrong though.

Rust makes deep copies explicit, whereas those languages don't. So if you program in the exact same style, it will be a bit more verbose; Rust is exposing a cost here.

That said, idiomatic Rust does lean a bit more functional than many languages.

Instead of using unsafe code, I would use RefCell first: «a mutable memory location with dynamically checked borrow rules»

https://doc.rust-lang.org/std/cell/struct.RefCell.html

Using clone() also often helps. clone() has an overhead since you copy bytes, but sometimes you need to get something working right first, then optimise.

There are other tricks if the borrow checker interferes.

I found that my brawls with Rust's borrow checker ended when I made ownership the focus of my code design. Coming from a C++/Objc/Go background I was used to creating an object on the heap and holding a reference counted/garbage collected pointer to the object wherever it was used. This is shared ownership of state, a style of coding Rust considers to be so egregious that it is a compile error.

Initially I would use constructs such as Rc<Box<T>> in an attempt to write Rust code in my old style. It took some time for me to realise that Rust's borrow checker was not just designed to avoid occasional data races, but shared ownership of state as a whole, and the problems it causes. Once I ceased trying to resolve each localised problem with the borrow checker, and I designed my code such that each and every object has a clear owner responsible for creating, maintaining and destroying it at all times during execution, I have had no further problems.

At the point you "get it", it is worth comparing the code you eventually wrote to the code you would have written initially. For me, it seemed that Rust's borrow checker had forced me to code in a better, safer style, rather than brainwashing me into thinking it had, so I have continued using Rust.

>" Coming from a C++/Objc/Go background I was used to creating an object on the heap and holding a reference counted/garbage collected pointer to the object wherever it was used. This is shared ownership of state, a style of coding Rust considers to be so egregious that it is a compile error."

Is the idea that because anyone else can come along and make reference to the same object on the heap mean "shared" in this context? If so would the opposite of "shared" be "owned" or "owned exclusively"? Is this what borrow checker is "checking" then?

In C++ the closest equivalent is probably unique_ptr [1], which only lets one reference exist at any given time and destroys the pointer when it goes out of scope, but also allows transferring the ownership to someone else. The C++ equivalent of Rust's Rc (reference counted pointer) is std::shared_ptr.

[1] http://en.cppreference.com/w/cpp/memory/unique_ptr

Thanks for the responses, these are really helpful.

Exactly. You can have as many immutable references to an object as you want, or you can have one mutable reference to it, not both. (Each of those references is how you borrow something from the real owner, hence the name.)

The borrow checker is that part of the compiler that makes sure that these borrowing rules are followed. Other rules are implemented by other parts of the compiler; the type system has the job of making sure you don't try to write to something that you only have an immutable reference to, for example.

I come from a C++ background, and I quite happily used references, const references, and move operations regularly. I dare say, a good bit of the code I wrote was a C++-analogue to the Rust code I write. It's just that the mutability differences reduce the code I have to look at considerably (e.g. instead of "const &" I simply have "&" and instead of "&" I have "&mut"--which reduces the quantity of code considerably).

I have been programming with C++ before the C++-0x and later standards introduced some of these concepts (e.g. move) and since, and the move to Rust (as far as the borrow checker is concerned) has been mostly smooth.

I think "I come from C++" isn't as important as the kind of C++ one has been used to.

Basics:

When you're writing (or read someone else's) functions, you should be considering three types of parameters:

  * (&)     borrowed
  * (mut &) mutably borrowed
  * ()      moved

The borrow operator should be your default/go-to decoration. Don't use the more destructive exchanges until the compiler forces you to. As I write this out, I'm thinking this might be a small short-coming in Rust's design. I.e., read/borrowed should maybe be the default/undecorated behavior and special pains should be required to move a reference.

Anyway, borrowed means, "I just wanna read some stuff off of that. I promise not to change a thing!"

Mutable borrowing means, "Gimme that--I'm going to change it! Expect it to differ as part of the output of this function."

Moving means, "That's mine now! I'm going to wreck it beyond any usage you'd try to do after I'm done. If you think you still want it, better hand me a clone."

It might also be good to spend some time working through the book if you haven't already: https://doc.rust-lang.org/stable/book/

Here're some HN comments about advanced patterns:

  * https://news.ycombinator.com/item?id=13470592#13470904
  * https://news.ycombinator.com/item?id=13470975
    * Also recommends http://cglab.ca/~abeinges/blah/too-many-lists/book/

Also, save and recompile early and often. If you're using IntelliJ IDEA [Community Edition] with the Rust plugin, compiling is just ctrl + r .

And if compile times are getting you down, you can still get the checks without producing a binary and save quite a bit of time with https://github.com/rsolomo/cargo-check

That's built in now!

Is the borrow syntax (&x) a reference while vanilla (x) is by value? Or are both passed by reference but with different ownership affects?

Reason I'm asking is I got hung up the other day on passing a String to a function that accepted &str which someone explained to me Strings dereference to &str but I think I just ended up more confused.

> Is the borrow syntax (&x) a reference while vanilla (x) is by value? Or are both passed by reference but with different ownership affects?

This is briefly addressed in the FAQs:

"What is the difference between passing by value, consuming, moving, and transferring ownership?

These are different terms for the same thing. In all cases, it means the value has been moved to another owner, and moved out of the possession of the original owner, who can no longer use it. If a type implements the Copy trait, the original owner’s value won’t be invalidated, and can still be used."

https://www.rust-lang.org/en-US/faq.html#what-is-the-differe...

There are more details in the chapter on the stack and the heap from the book (https://doc.rust-lang.org/book/the-stack-and-the-heap.html)...

"The stack is very fast, and is where memory is allocated in Rust by default."

"What do other languages do?

Most languages with a garbage collector heap-allocate by default. This means that every value is boxed. There are a number of reasons why this is done, but they’re out of scope for this tutorial. There are some possible optimizations that don’t make it true 100% of the time, too. Rather than relying on the stack and Drop to clean up memory, the garbage collector deals with the heap instead."

So unless your data structure is boxed, it's allocated on the stack and passed by value and different ownership effects apply as well.

> I got hung up the other day on passing a String to a function that accepted &str which someone explained to me Strings dereference to &str but I think I just ended up more confused.

String literals de-sugar to &str. E.g.,

  fn borrow_str(s: &str) {}

  borrow_str("foo");         // This works
  borrow_str(String::new())  // this doesn't work

  fn take_str(s: String) {}

  take_str("foo")            // Doesn't work
  take_str(String::new())    // works
  take_str("foo".to_owned()) // works

Tiny bit: string literals don't desugar to str, that is their actual type.

&String will coerce to &str thanks to Deref coercions.

"I keep end up fighting the compiler with borrowing errors."

To set expectations, I tell everyone that I've encouraged to try Rust that they will hit this same problem, then hate themselves and want to give up on computers and live in a cave. Once they understand that they aren't just going to "get" Rust in four hours reading the book while Seinfeld is on in the background, I tell them that when they get frustrated with the borrow checker, just re-read the "Ownership" and "References and Borrowing", "Lifetimes" chapters from the book and then rinse and repeat. I tell them that by the fourth or fifth time it will "just click". So far, so good.

I know this isn't the most ideal method, but until it's cemented, you don't really understand the mechanics of what you're able to get away with. Persist... we've all gone through your frustrations, but can see the awesome bright light at the end of the tunnel (no, it's not a train).

> I'm really not sure where to turn for help.

I get most of my help on #rust on IRC. But for those who don't prefer that the discourse site https://users.rust-lang.org/ is probably idea. Or if you prefer reddit, do /r/rust -- they have a weekly sticky thread for questions IIRC.

The community is super friendly so "I'm really not sure where to turn for help" should be easy to solve. IMO just keep asking questions until you can internalize the rules you're getting help with.

>It seems like everything I know about structuring a program goes out the window when borrowing comes into the picture.

I know squat about borrowing rules, but why not just toss things on the heap (is that 'Arc'?) until you get better?

BTW -- do you already have some familiarity with 'C'? IMO that might be a simpler place to start. It has virtually none of the borrowing rules but perhaps you could quickly find out why they exist in Rust.

Yes. OP, please drop by IRC, the forums, or Reddit. We are extremely happy to work through issues and help explain what's going on, generally. Half the time during US working hours you'll literally get me.

Here's my understanding. Please correct me if I am wrong. Visualizing your program as a tree of values that are borrowed, owned, mutated through variables helps in reasoning about the borrow checker

When a variable owning a value goes out of scope all the children goes with it. Remember in rust, there can only be one owner to a value (Rc and Arc variables lets you have multiple owners but that's for special cases). Therefore, if you're passing a variable to a function you should either

1.) transfer ownership to the function being called.

2.) have the function being called borrow the value through references. Since this doesn't transfer any ownership you can have multiple references to the same value. The value and its children (think of a Vec<Animal>) are taken down from the tree it belongs when the variable owning this value goes out of scope.

That's the cool thing that does rust does. The compiler knows when to free things just by following this one rule. A value should have only one owner. The responsibility of managing memory is now taken by the compiler instead. Hence no dangling pointers or segmentation fault unless you use `unsafe` blocks where this assurance isn't valid anymore.

Visualize your program to be manipulating these multiple trees made up of diverse types.

I've been doing software dev since I was a kid. In my mid-30s now. I spent almost a month with Rust about a year ago. I was very disappointed. The borrowing rules seem to have changed as the language had evolved. I found the Internet documentation to be very confusing. My conclusion was to let Rust be. It felt like coding with restraints, and as my Rust expert friends tell me, that is the way it is supposed to be. My friends write kernel modules, do low level systems etc. where they know exactly what they are doing. I mostly do exploratory programming. I think Go, Elixir and Python are the best languages for the kind of stuff I'm interested in.

Interesting. I wonder what most people's thoughts are on the market/target area for Rust. My complete outsider perspective was that Rust was a "replacement" for C, C++, Ada, etc., and that "most" programs would be better suited to be written in higher level languages.

Whenever someone asks this question, I point them at this...

Learning Rust With Entirely Too Many Linked Lists: http://cglab.ca/~abeinges/blah/too-many-lists/book/

The point of this tutorial is not "wow it's hard to write linked lists in Rust", but that Rust supports a number of different memory models, and this tutorial takes you through each of them by writing a linked list using each one.

Additionally, it's snarky and entertaining.

All that said, I think it's a really good crash course on the Rust memory model, and once you have your brain wrapped around the various approaches, it should hopefully make it easier to understand how to avoid encountering those borrow checker errors.

Do you understand how to write code in, say, C? Like do you understand the scope of manually managing memory, not making memory safety errors? When I run into borrow checker errors I try to think through exactly who owns what and who will free it when.

I'd suggest having a look at the new (unfinished, but that affects mostly the later chapters) rust book. Here's the ownership chapter: http://rust-lang.github.io/book/ch04-00-understanding-owners...

Object ownership varies widely by language (I just wrote up a survey of why - https://codewithoutrules.com/2017/01/26/object-ownership/), and no one ever teaches object ownership as an explicit skill (it's always implicit with each language's semantics). So it's pretty natural that switching styles would be difficult.

Perhaps if you understood why these are rules are in place, or compare to other rules from other languages? GNOME project has nice strict rules for C: https://developer.gnome.org/programming-guidelines/unstable/...

(Full disclosure: never written any Rust, have just skimmed docs.)

Your post might possibly drown soon here, so..

> I'm really not sure where to turn for help.

both StackOverflow and reddit.com/r/<yourproglanguage> have rarely if ever failed me..

Hey, I also started learning Rust not so long ago (~5 months).

By the sounds of it didn't have quite as much trouble as you're experiencing.

Having said that, I found experienced Rust developers grossly under estimate how hard it is for newcomers to the language (having asked questions on Reddit's r/rust which gave the impression these are some concepts developers can learn, then move on without too many troubles).

A re-occurring problem I found is while everything I could read made sense on paper - composing concepts to make real-world, you run into _many_ random problems that you don't even have the vocabulary to properly question or troubleshoot.

I found this works well:

* Ask general big-picture questions on Reddit.

* Ask spesific technical questions on StackOverflow.

* Ask various short/general questions on IRC (#rust).

As for fighting the borrow checker and compiler errors generally - ask some more experienced Rust developers if you're taking the wrong approach entirely. If this project is too complicated, try pick some simple projects where you're not continuously running into borrow checking errors. You'll still hit them from time-to-time, but then at least you're not overwhelmed by compiler errors.

It seems like everything I know about structuring a program goes out the window when borrowing comes into the picture.

You can substitute concurrency for borrowing. Congrats. You're on the path towards understanding another paradigm. You're on your way to expanding your mind and growing a bunch of new neural connections. The learning curve means the eventual triumph will be that much better.

It really depends on what you are trying to do. Spending a month fighting the borrow checked doesn't seem normal to me.

If you are implementing data structures and stuff, you might want to read more about the actual limitations of the borrow checker. Some kinds of code just can't be done with the constraints of the borrow checker. Embrace unsafe blocks and then gradually try to move as much code as possible away from those unsafe blocks. Try to think of "unsafe" as merely "unverified by the compiler". There is nothing inherently evil about unsafe.

If you are doing more application-level programming, I recommend you to get acquainted with the functional style of programming, if you haven't done this yet. Pick a more conventional language and then try to implement something using as few assignments as possible. Get used to and embrace immutability. Once you are used to immutability, Rust will be much easier too.

It is likely that your structure is in fact allowing _potential_ errors to enter into the picture. Most of the time in C or C++ you would ignore them since it is obvious to you where things will be owned and not owned as the program progresses, but the compiler is must more strict and it will not allow any potential errors to exist. I don't think you will find a shortcut around identifying those areas until you work through them a bit.

The other big thing that I wrestled with was my idea of in place mutation of data, and how much rust hates that. It took a while for me not to feel so bad about making a local variable to maintain ownership of some memory and "moving" it in and out of some parent structure. While it might look bad, there is a deal of trust you need to have with the compiler that it will in fact optimize out those move, and your main goal is to make sure ownership is explicit.

Well, I think a couple of points:

1. An outline of the problem you're trying to solve helps 2. You should really stop by the #rust-beginners channel on IRC. Everyone is super friendly

Glad to see that you're trying Rust, and if I'm there when you stop by - glad to help if I can!

Try a "back-to-basics" approach of writing less structured code, as in, factor it less, use more plain data structures, just write straight-line imperative blocks of code. This is actually the path of least resistance in most languages, but they all offer shiny features and don't stop you from overcomplicating it by prematurely distributing functionality into abstractions.

Read more open source code. I'm not sure what project to recommend, maybe some of the https://tokio.rs related projects? (e.g. https://github.com/tokio-rs/tokio-core).

Try these screen casts: http://intorust.com/

That `into` is so evil, I could not be used again if I'm `into_rust()` :(

    pub fn as_rustacean<P, R>(&mut person: P) -> &mut R where P: Programmer, R: Rustacean;

Reply on news.ycombinator.com