Rust 1.46
blog.rust-lang.org
blog.rust-lang.org
In addition to the linked examples, I have some code of my own which is made simpler due to this feature: https://github.com/RustAudio/ogg/commit/b79d65dced32342a5f93...
Previously, the table was present as an array literal in C-style, now I can remove it once I decide for the library to require the 1.46 compiler or later versions.
Link to the old/current generation code: https://github.com/RustAudio/ogg/blob/master/examples/crc32-...
This sounds promising. Can you give examples? I don't know Rust at all, and the reason I like C++ is its metaprogrammability.
I don't know constexpr well enough to comment on that claim.
> All boolean operators except for && and || which are banned since they are short-circuiting.
I guess I'm missing something obvious but why does the short circuiting break const-ness?
(I'm a little surprised they weren't stabilized at the same time! Edit: they were! I just didn't look closely enough.)
They were stabilized at the same time, see the release announcement.
Until this version of rust, all conditional branches were banned from const functions.
I guess to keep things simple they just banned any feature that might cause branching.
That said, coming from a FP background (mostly Haskell/JS, now TS) Rust is... hard. I do understand the basic rules of the borrow checker, I do conceptually understand lifetimes, but actually using them is tricky.
Especially in a combinator world with lots of higher order functions/closures it’s often completely unclear who should own what. It often feels my library/dsl code needs to make ownerships decisions that actually depend on the usage.
Anyways, I guess this gets easier over time, right? Should I avoid using closures all over the place? Should my code look more like C and less like Haskell?
[edit] great answers all, providing useful context, thanks
Yes.
> Should I avoid using closures all over the place?
Not necessarily.
> Should my code look more like C and less like Haskell?
Yes. Others sometimes don't like to hear this, but IMO, Rust is not at all functional. Passing functions around is not ergonomic (how many function types does Rust have again? Three?). Even making heavy use of Traits, especially generic ones, is difficult.
Rust is very much procedural. Java-style OOP doesn't work because of the borrowing/ownership. And FP style function composition doesn't work without Boxing everything. But then you'd need to be careful about reference cycles.
It has to, right? ATS has many function types as well, plus stack-allocated closures (I think Rust has that too??)
(and yes, there are three types of closures, because they need to know if they take said struct by reference, by mutable reference, or by owner.)
Depending on what you meant, there are more than three:
* There are 3 traits, used by closures depending on their needs:
* Fn(Args) -> Output
* FnMut(Args) -> Output
* FnOnce(Args) -> Output
* *Every* `fn` is its own type (`fn() {foo}`)
* Function pointers (`fn()`), which is how you pass the above around in practice
I think this is like saying Python is very much procedural: true, but loses some nuance. Rust has some attributes of OOP, some attributes of FP. Some constructs from OOP and FP are made harder once you involve borrowing. Saying it is procedural conjures images of Pascal and K&R C in people's minds. To bolster your argument, though, I mostly use method chaining for iterators but every now and then I need to turn it into a `for` loop to keep the lifetimes understandable for the compiler, myself and others.
But anyway, I don't really disagree with your point about categorizing languages as OOP, Procedural, or Functional.
But honestly, in this case, I think it's pretty damn clear than Rust is procedural WAY more than it's either OOP or FP. (Note: By OOP, I mean Java-style with tall ownership hierarchies and object-managed mutable state, not necessarily caring about inheritance. And definitely not referring to Alan-Kay-Style-OOP a la Lisp and Smalltalk).
Scala can be looked at as FP and/or OOP. C++ can be looked at as Proc and/or OOP. Python, IIRC, can kind of do all of them, but I don't remember it being easy to make copies/clones in Python, so FP is questionable.
Have you ever tried to write two versions of a complex async function in Rust? One with async and one with Futures combinators? Due to ownership, the Futures combinators approach very quickly devolves into a nightmare. The language doesn't "want" you to do that.
What about function composition? Very awkward to do with matching up the different Fn traits.
And deeply nested object hierarchies are a no-go, too, because of the inability to do "partial borrows" of just a single field of a struct.
I mean, yes, it's not C because it has a real type system and generics. But... it's pretty much C in that you just write functions and procedures that operate on structs.
EDIT: Perhaps my "hardline" approach on calling Rust procedural is in response to people who have come to Rust from non-FP languages, see `map`, `filter`, and `Option` and start calling Rust functional. That's not functional programming! Ask someone who does OCaml to try out Rust and see if they call Rust functional afterwards.
I'm not totally sure if this is what you mean, but FYI you can borrow multiple fields mutably using destructuring:
struct Foo(u8, u8);
fn main() {
let mut bar = Foo(1, 2);
let Foo(ref mut x, ref mut y) = &mut bar;
*x += 1;
*y -= 1;
println!("{} {}", bar.x, bar.y);
} foo(&bar.x, &bar.y, &bar.z)
foo(&bar)This is also why it's okay in the body of a single function; that doesn't impact a boundary.
We'll see what happens in the future.
impl Foo {
fn bar(self: Foo { ref a, mut ref b, .. }) {}
}
That said, I think putting it in the argument list like that is a terrible idea. It would add far too much clutter. What if it was put into the where section, sort of like this:
impl Foo {
fn bar(&mut self, other_param: &Data)
where self: borrows{ a, b, .. },
other_param: borrows{ a, b, ..},
{
// Function body
}
}
So it often comes up that you might call several methods in a given scope. If even one of those mutably borrows one field of one sub-object, then you can't have any other borrows of that object anywhere else in that scope.
Newbies from other languages trip on that often enough that I used to see questions about it in r/rust fairly frequently.
Scala can be looked at ...
Interesting perspectives, and I largely agree with all of them.
Related: I heard someone else say that while Clojure and Erlang embrace immutability for concurrency, Rust shows that you can "just mutate". It's still safe for concurrency (due to its type system).
Rust seems to be one of the only languages that embraces the combination of algebraic data types + stateful/procedural code.
But I've also found this in my Oil project [1], which is written with a bunch of custom DSLs!
I wrote it in statically-typed Python + ASDL [2], so it's very must procedural code + algebraic data types. Despite historically using an immutable style in Python, this combo has grown on me. Lexing and parsing are inherently stateful, and use a lot of mutation.
----
On top of that, my collection of DSLs even translates nicely to C++. Surprisingly, it has some advantages over Rust! The model of algebraic data types is richer ("first class variants"), described here:
https://news.ycombinator.com/item?id=24136949
https://lobste.rs/s/77nu3d/oil_s_parser_is_160x_200x_faster_...
Yes! I will repeat a sentiment I articulated on Reddit about that. Even after having used Rust on a handful of small-to-medium sized projects since 2016, I never realized that I could loosen/abandon my immutability fetish that I had been trained to love over the years of working with C++ and Java. C++ needs it for concurrency, and Java needs it for concurrency and because every method can mutate its inputs without telling you. Rust doesn't have either of those problems. Having immutable-by-definition objects in Rust isn't really that useful (unless, of course, the thing is naturally, semantically, immutable anyway, like a Date IMO). It was an eye-opening epiphany and I'm excited for my next Rust session to see how my "new worldview" pans out. :)
Does Rust have something like C++'s const methods? Where you can have a method that mutates a member, but doesn't logically mutate from the caller's perspective?
It seems like you could be prevented from having races on individual variables, but still have races at a higher level.
Like on database cells. I guess no language will help you with that, and that's why Hickey wrote Datomic -- to remove mutability from the database.
Yes! "Interior mutatbility" is the term to search for. In Rust, you'd wrap the field in a RefCell<T>. Many connection-pool implementations use interior mutability to manage the connections transparently to the caller.
Interior mutability is basically what, e.g. OCaml, does by default. In Rust, it's opt-in.
Yeah, DB ops are always a sticking point for figuring out how to write my APIs.
Those people who think that "FP" means "type system like Haskell" are wrong, though, IMO. It precludes languages that are much more function-based, such as Clojure, Schemes, Elixir.
When people say "OOP" or "class-oriented-programming" are you saying that they're referring to implementation details such as memory allocation?
It ends up being doable. I dabbled in ATS, developed Stockholm syndrome, and now Rust ain't too bad.
Higher-order functions are difficult in Rust or with linear/affine types in general. Haven't looked at what Rust does recently.
> Should I avoid using closures all over the place? Should my code look more like C and less like Haskell?
When in Rome do as the Romans :)
Anyway, some fun imperative programming stuff you can do in Rust that is fickle in Haskell (or OCaml/Standard ML).
* If you need to keep unchanged the input, you must either use a reference-to (.iter()) or copy-of (.iter().cloned()) of each item
* If you don't need the input ever again, you should move the items (.into_iter())
These rules follow for each step of the chain.
I very very often write very Functional code in Rust and I find it natural and easier to reason about than imperative-style code. The example I could find the fastest: https://github.com/thenewwazoo/aoc2019/blob/master/src/day10...
Edit: another example (this one uses types that are Copy so the copies are implicit) https://github.com/thenewwazoo/cryptopals/blob/master/src/tr...
Another edit: I am not a Functional programmer, and have never known Haskell or any Lisp. Erlang is as close as I've ever gotten. I've found Rust to be a fantastic language for writing Functionally.
1) if perfomance isn't a measurable problem for you, then there's on point on eking the last bit of performance from these allocations
2) it simplifies the code itself
3) sometimes clones are actually efficient, people forget to make their small ADTs Copy
4) if you're learning the language this lets you delay the moment when you have to fully understand the way lifetimes actually behave in complex cases, which means that when you do do that you will have a better grasp of the rest of the language and will be able to form a better mental model of how it fits with other features
There is a lot to like, understand lifetimes conceptually, but it's hard.
I've been using Rust for a little over year, almost daily at work, and for several projects. I have a pretty good intuition about how the borrow checker works and what needs to be done to appease it. That said, I don't think I'm any closer to understanding lifetimes. I know conceptually how they are supposed to work (I need the reference to X to last as long as Y), but anytime I think I have a situation that could be made better with lifetimes, I can't seem to get the compiler to understand what I'm trying to do. On top of that very little of my code, and the code I read actually uses lifetimes.
A simple example i often run into is wanting to do something with a string, without taking owned parts of the string. Very intuitive how the str matches the lifetime of the owned value.
On the otherhand, the other day i was trying to write a piece of software where:
1. I wanted to deserialize a large tree of JSON nodes. I had the potential to deserialize these nodes without owning the data - since Serde supports lifetimes, i could deserialize strings as strs and hypothetically not allocate a lot of strings.
2. In doing that, because a tree could be infinitely large i couldn't keep all of the nodes together. Nodes could be kept as references, but eventually would need to be GC'd to prevent infinite memory.
3. To do this, i _think_ lifetimes would have to be separate between GC'd instances. Within a GC'd instance, you could keep all the read bytes, and deserialize with refs to those bytes. When a GC took place, you'd convert the remainder partial nodes to owned values (some allocation) to consume the lifetime and restart the process with the owned node as the start of the next GC lifetime. ... or so my plan was.
I have, i think, just enough understanding of lifetimes to _almost_ make that work. I _think_ some allocations would be required due to the GC behavior, but it would still reduce ~90% of allocations in the algorithm.
Unfortunately, i got tired of designing this complex API and just wrote a simple allocation version.
Conceptualizing allocations and the lifetimes to make it work are.. interesting. Especially when there is some data within the lifetime that you want to "break out of" the lifetime, as in my example (where i had a partial node, and i made it owned).
I still think i understand enough to do it - it'll just take a fair bit of thinking and working through the problem.
When the compiler starts complaining about lifetimes issues, I tend to make everything clone()able (either using Rc, or Arc, or Arc+Mutex, or full clones).
Because if you start introducing explicit lifetimes somewhere, these changes are going to cascade, and tons of annotations will need to be added to everything using these types, and their dependent types.
Now that's damning with faint praise.
It is definitely not easier compared to C++, contrasting with D, which is easier than C++.
However, the program worked correctly at the first try, which I guess it is also a consequence of the Rust model.
example:
fn add(mut self) -> Self { self }
fn add(self) -> Self { self }
instead of:
fn add(&mut self) {}
fn add(&self) {}
With this, you will be able to ‘store’ closures easily and apply them later. No more fighting with the borrow checker over where to borrow as mut or not. You will also avoid a few copies.
Are there any particular set of problems that I can solve systematically, so that I can learn all the features of Rust?
https://doc.rust-lang.org/stable/rust-by-example/ is the "by example" introduction, which is all about sample programs, but feels a bit dated, IMHO. Still not incorrect, but not up-to-date.
You may also like the O'Reilly book, or Rust In Action, which use more fully-featured example programs more heavily than The Book does.
The manual is safer even though it's harder to find your exact problem and solution, especially when you're just starting out.
As always, feel free to drop into the Rust Stack Overflow chat room[1], or any of the official Rust discussion channels, and ping me or other Stack Overflow contributors to review and update answers.
Question for Rust experts: On what ETL tasks would you expect Rust to outperform Numpy, Numba, and Cython? What are the characteristics of a workload that sees order-of-magnitude speed ups from switching to Rust?
Now, if you're doing lots of computation in Python itself - not within the confines of Numpy - that's where you might see a significant speed boost. Again, I don't know precisely how Rust and Cython would compare, but I would very much expect Rust to be significantly faster, just as I would very much expect C++ to be significantly faster.
With rust you can stream each record and leverage the insane parallelism and async-io libs (rayon, crossbeam, tokio) and a very small memory footprint. sure you have asyncio in python but that’s nowhere near the speed of tokio.
That way you leverage a more developed data ecosystem, can call python when necessary and avoid writing low level code.
Depends on the task of course.
I guess I generally do like the pythonic paradigm of an interpreted glue language orchestrating precompiled functions written in other languages. I don’t need or want to compile the entire pipeline end to end after every edit, that slows my development iteration cycle times.
I just want to write my own fast compiled functions to insert into the pipeline on the rare occasions I need something bespoke that doesn’t already exist in the extended python ecosystem. It seems like a lower level language would be optimal for that?
If you just need to fill a small and slow gap maybe something like numba is also a good option to stay within python.
Going all the way to a low level language would require the compilation, the glue code and expertise in both languages. Probably that slows down the development pipeline more than the JIT compilation from julia or numba.
Anyway, any opportunity to learn/practice some rust is also great!
It also taught me about Boxes and Rc's, which are essential for certain kinds of things, and which I don't remember being covered in the main Rust Book at all
What I like about the latter is how closely it steps through the problem-solving process within the context of a very familiar task, teaching you at each stage 1) why the borrow-checker is upset and 2) what tool you need to apply in order to satisfy it. If the Book taught me "what is Rust and what are its features?", this taught me "how do I use Rust in practice?".
I found that approach for Rust in particular to not work well at all, and have colleagues who've reported the same. There are some fairly complicated, fundamental concepts that are unique to Rust that I think need to be tackled before you can really do much of anything (mostly borrowing and lifetimes), and that's not immediately obvious from starter programs -- because of lifetime elision, some early programs can look deceptively familiar, but there's a bunch of barely-hidden complexity there, and as soon as you start to stray from the tutorial path, you'll run headfirst into a wall of compiler errors that you're not yet equipped to understand. For Rust I'd highly recommend just reading a book cover to cover first (either TRPL or the O'Reilly one), and then starting to write code.
It is the Rust way of specifying a function as being _pure_. In other words the output is dependent only on the function arguments, and not on any external state.
This means they can be evaluated at compile time. I suppose in the future, it could also allow better compiler optimizations.
If you say that 'pure' functions are not compile-time-evaluatable because they may be given parameters that are not known at compile time, then you must also say that const fns are not compile-time-evaluatable. I think it's also clear that we mean for const fns to count, so the assumption that the parameters are known at compile time was implicit in the question.
Under those two assumptions: are pure functions evaluatable (in theory) at compile-time (on values known at compile time)? As far as I can think, the answer is yes? In which case, I'm not entirely sure what the distinction between 'pure' and 'cosnt fn' is supposed to be except to separate out the part of 'pure' functions that can are evaluated in practice. Is there anything more to it?
const functions can't directly do any IO or even allocation - at the moment.
But this can be easily circumvented, eg by using a proc macro that does IO.
Sidenote: even in Haskell the function signature doesn't guarantee purity, due to unsafePerformIO.
Sadly it looks like the wayback machine does not have a copy of the original. Does anyone know how to get one?
The URL is: http://thread.gmane.org/gmane.comp.lang.rust.devel/3674/focu...
I've also seen it without the trailing /focus.
The web interface of gmane.org is down, so the link is not available. Turns out though that the rust-dev mailing list archive is present on both mail.mozilla.org and mail-archive.com, so one only has to find the mail corresponding to the link.
https://www.mail-archive.com/rust-dev@mozilla.org/
https://mail.mozilla.org/pipermail/rust-dev/
Using archive.org's "find all archived websites with this prefix" feature one can obtain a total of three archived e-mails.
http://web.archive.org/web/*/http://article.gmane.org/gmane....
http://web.archive.org/web/20140723013539/http://article.gma...
http://web.archive.org/web/20140719142224/http://article.gma...
http://web.archive.org/web/20141225073140/http://article.gma...
Now, one searches for lines in those e-mails in on mail-archive.com and finds these corresponding links:
https://www.mail-archive.com/rust-dev@mozilla.org/msg06831.h...
https://www.mail-archive.com/rust-dev@mozilla.org/msg09516.h...
https://www.mail-archive.com/rust-dev@mozilla.org/msg10494.h...
Observe that the differences between the two IDs are different each time, namely decreasing: 53, 50, 45
So it's not a constant difference. What to do now?
Let's google the URL! It points towards this hn comment: https://news.ycombinator.com/item?id=7554676
It gives one piece of information: the e-mail was written by Graydon. Similarly, commenters in https://www.reddit.com/r/programming/comments/1t8y6g/why_rus... mention his name, making it very likely that the e-mail was written by him.
Another hint comes from the reddit thread you linked above: someone named maxcan stated they started the thread. Looking up their name plus "pure" gives only e-mails from a single thread, including an e-mail from Graydon: https://www.mail-archive.com/search?l=rust-dev%40mozilla.org...
This is the E-Mail:
https://www.mail-archive.com/rust-dev@mozilla.org/msg03913.h...
https://mail.mozilla.org/pipermail/rust-dev/2013-April/00392...
Also archived it, just to be sure:
http://web.archive.org/web/20200827181214/https://www.mail-a...
It covers precisely the topic you mentioned and is in a thread started by maxcan. I think it's the e-mail we are looking for.
To verify, the difference between the two IDs is either 239, or 58, depending on which of the two numbers in the URL point to the actual E-Mail, but 58 is more likely. The 0.7 release announcement for example has a difference of 57 and is quite close to both:
https://www.mail-archive.com/rust-dev@mozilla.org/msg04653.h...
This is! Thank you so much!
(Niko also once wrote a blog post which gives an overview of the old purity system: https://smallcultfollowing.com/babysteps/blog/2012/10/12/ext... )
Thanks for that link to Niko's blog too!
In that case, you currently have to write code like:
if let (Some(username), Some(password)) = (username, password) {
/* both are set */
}
else {
/* at least one is not set */
}
let (username, password) = username.zip(password).ok_or("one or more required parameters is missing")?;
let (username, password) = username.and_then(|username| password.map(|password| (username, password))).ok_or("one or more required parameters is missing")?;
Therefore I would actually prefer the long/verbose form without the zip.
"if let" is certainly a special construct - but it's also one that Rust authors and reviewers will typically encounter rather fast since a lot of error handling and Option unwrapping uses it. Knowing the majority of library functions will imho take longer.
The upside of course is that any computation you compute at compile time is a computation that you don't compute at runtime. For some applications this trade off is definitely worth the cost of admission.
At the end of the day it's a trade off that will have to be made in light of the scenario it's being used in. Being able to make that decision is a good thing.
One thing that people may not realize, especially now that we have loop. You may expect this to hang the compiler:
const fn forever() -> ! {
loop {
}
}
static FOO: u32 = forever();
error[E0080]: could not evaluate static initializer
--> src/lib.rs:2:5
|
2 | / loop {
3 | |
4 | | }
| | ^
| | |
| |_____exceeded interpreter step limit (see `#[const_eval_limit]`)
| inside `forever` at src/lib.rs:2:5
That could be a real hard-to-source build stall.
> while, while let, and loop
> the && and || operators
Common Lisp user here. Why just that? How come you can’t have the entire language as well as all your language customizations available at compile time for evaluation?
Why isn’t `const fn` like this too? One word answer: determinism. Rust takes type/memory/access/temporal safety very seriously, and consequentially you can’t use anything in a `const fn` that isn’t fully deterministic and doesn’t depend in any way on platform-specific behavior. This includes, for example, any floating-point computation, or any random number generation, or any form of I/O, or handling file paths in an OS-specific way, or certain kinds of memory allocation. The span of things possible in `const fn`s has been expanding over time, and will in the nearish future largely overtake C++’s direct counterpart of it (`constexpr` functions) in capability. But some things will intentionally never be possible in `const fn`s, for the reasons given above.
I can't wait for when we'll be able to `const fn` all the things. Regex, expensive constants that feel as though they should be literals, etc.