Was Rust Worth It?
jsoverson.medium.com
jsoverson.medium.com
I know rust gives memory safety and how important that is, but the ergonomic is really bad. Every time I write some rust I feel limited. I always have to search libraries and how to do things. I cannot just "type the code".
Also the type system can get out of control, it can be very hard to actually know what method you can call on a struct.
I still think rust is a great tool, and that it solves tons of problem. But I do not think it is a good general purpose language.
> But I do not think it is a good general purpose language.
Remember that this is not a sentiment that's shared by everyone. I use Rust for tasks that need anything more complicated than a shell script. Even my window manager is controlled from a Rust program. I say this as someone who has been programming in Python for nearly two decades now. At this point, I'm about as fast in Rust as I am in Python.
While the Rust devs believe such cyclic references are rare - I think this speaks mostly to the problem domain they are focused on. Relational models are everywhere in Apps, they are often common in complex systems software like databases, and they are fairly rare in firmware/drivers/system code code.
There are a few patterns for dealing with cyclic references, but they all end up requiring either unsafe or a main "owner" object which you clean up occassionally (effectively arena allocation). Having now worked in C/CPP - the idea of having unsafe blocks sprinkled around the code doesn't bother me, and many C/CPP components have some form of arena allocation built-in. I just wish Rust learning resources would be more upfront about this.
While beginner resources don't dwell too much upon cyclic references, they don't consider unsafe blocks as unusual. All the material I've seen say that there are certain domains where Rust's compile-time safety model simply won't work. What Rust allows you to do instead, is to limit the scope of unsafe blocks. However, the beginner material often won't give you too much details on how to analyze and decide on these compromises.
Anyway, compile-time safety checks (using borrow checker) and manual safety checks (using unsafe) aren't the only way to deal with safety. Cyclic references can be dealt with runtime safety checks too - like Rc and Weak.
Indeed. Starting out with code sprinkled with Rc, Weak, RefCell, etc is perfectly fine and performance will probably not be worse than in any other safe languages. And if you do this, Rust is pretty close to those languages in ease of use for what are otherwise complex topics in Rust.
A good reference for different approaches is Learn Rust With Entirely Too Many Linked Lists https://rust-unofficial.github.io/too-many-lists/
It is no accident that while Val/Hylo, Chapel and Swift have taken inspiration from Rust, they have decided to not inflict the affine types directly into the language users, rather let the compiler do part of the work itself.
It's not like that you can't write relational models in the safe Rust. The only forbidden thing is a reference pointing arbitrary memory, which is typically worked around via indices and often more performant in that way. It is much rarer to find applications that need an absolutely random pointer that can't be hidden in abstractions in my opinion.
I've still been meaning to write something in / learn Rust's ways of thinking; is there not an intended replacement for these data structures? Or do they expect it all to go under Unsafe?
For linked-lists, there's one in std and the majority of people should never have to write their own as it's error prone and requires unsafe.
For graph use-case then you can either use ECS, arena, ref counting or unsafe, but you're probably better off using/developing a dedicated crate that optimizes it and abstract it away behind an easy to use (and safe) interface.
Sometimes such a primitive doesn't exist and you should use unsafe yourself, but then you're the one supposed to make sure that your code is in fact sound. If you keep unsafe for small portions of the code you can reason about and extensively test so Miri gives you a good level of confidence, then it's fine to use unsafe. But it's the more expensive option, not the default.
a doubly linked list
Parent field is something where you have a clear hierarchy (it's not really “cyclic”, so it's the perfect use-case for weak references).
When coming from a managed-memory language, this obviously requires some conceptual effort to understand why this is a problem at all and how to deal with it, but when compared to C or C++, the situation is much better in Rust.
It's very noticeable how often people who "need" a linked list actually only wanted a queue (thus Rust's VecDeque) or even a growable array (ie Vec).
Aria has a long list of excuses people offer for their linked lists, as well as her discussion of the time before Rust 1.0 when she sent lots of Rust's standard library collections to that farm up-state but wasn't able to send LinkedList.
I just put up a blog post about this actually :) https://jacko.io/object_soup.html
> promoting the shared state to a container object for the set
Yeah I think that's a good way to describe these "ECS-ish" patterns.
They are.
I have not had to use cyclic references ever, except once doing experiments with fully connected graphs, that was very unusual
If you're doing a lot cyclic references Rust is not the right choice. Horses for courses
But are you sure you're using the best algorithm?
Doesn't it just come down to the question of who owns the node?
If it's a tree, and parents are never removed before children, just make the child owned by the parent and keep a weak reference to the parent.
If it's a general graph, and vertices can exist or not exist regardless of edges, keep a list of them independent of the edges, and keep weak references in the edges.
If it's a graph where a one or a few roots exist, and nodes exist as long as there's a path from a root node to them, that sounds like a classic use case for Rc<>.
Is there a common use case I'm missing?
https://jacko.io/object_soup.html
Agreed that I wish more beginner Rust books had a section about this. The pattern is quite simple, but it's hard for beginners who get stuck to realize that they need it.
https://www.kernel.org/doc/html/v4.18/driver-api/uio-howto.h...
And yes the Python interop is excellent.
I don't want to program in anything else anymore. I don't want to deal with obscure C++ error messages, C footguns and lack of ergonomics, I don't want to deal with abstraction hell of Java, or the poor person's typing that python has.
I have been programming in Python for the past 6 years, I know all sorts of obscure details, and with rust, I just don't need to think about all of those issues.
Not really surprising, given that you have C and C++ background. That's what I was trying to highlight. Rust isn't a confusing or unproductive language as many project it to be - if you have the conceptual understanding of what happens on the hardware. Especially about stack frames and RAII. If you know those, the borrow checker complaints will immediately make sense and you will know how to resolve them.
Add rust-analyzer (Rust's language server) to it, you get real-time type annotations and a way to match types correctly in the first attempt. In my experience Rust also helps structure your program correctly and saves a ton of time in debugging. All in all, Rust is a fast way to write correct programs.
What reading materials will help me level up my understanding of this?
I have reasonable understanding of "what happens on the hardware" (been writing kernel code for years), know modern C++ (with RAII and stuff) and Rust is confusing and unproductive language for me.
Some people pick up rust quickly because it clicks into their brain early, some take longer or end up bouncing off.
Back then, people were debating the design decisions that led to the borrow checker, in public for everyone to see (on Reddit and IRC). They were trying to avoid memory safety issues in Firefox and Servo. They were even a bit surprised to discover that the borrow checker solved many concurrency bugs as well.
Then I got into Haskell, and functional programming, that made thinking about traits, immutability, and all functional aspects a breeze.
Then finally, I got into rust again, use it at work and personal projects. Somehow I managed to rewrite a project that took me 4 months in Python in about 4 days. It was faster, more robust, cleaner, and I could sleep at night.
This is factually impossible.
For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
This is not even considering more local complexities, like data structures with cyclical references.
How do you outright deny something as subjective as my personal experience? Besides, I'm not the only one in this discussion that made the same opinion.
> For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
While GC'ed languages allow you to skip a proper initial design stage, it's a stretch to claim that it's not required at all. In my experience using Python, while the initial stages are smooth, such design oversights come back and bite at a later stage - leading to a lot of debugging and refactoring. This is one aspect where Rust saves you time.
> This is not even considering more local complexities, like data structures with cyclical references.
I'm not going to dwell on cyclical references, since there's another thread that addresses it. They point out a way to make it as easy in Rust as it is in GC'ed languages.
Meanwhile, the upfront architecture and data structure design isn't as complicated as you project it. Rust is mostly transparent about those - even compared Python. How well do you understand how Python manages Lists, dictionaries or even objects in general? I often find myself thinking about it a lot when programming in Python. While you need to think upfront about these in Rust, there's actually less cognitive overhead as to what is happening behind the scenes.
They didn't specify.
The only time I find I have to "fight the compiler" is when I write concurrent code, and you can sidestep a lot of issues by starting with immutable data and message passing through channels as a primitive. It's a style you have to get used to, but once you build up a mental library of patterns you can reasonably be as fast in Rust as you are in Python.
Every language requires this (if you want robust code), most just let you skip it upfront ... but you pay dearly for doing so later.
Having an editor with rust-analyzer running is massively helpful too since ownership issues get highlighted very quickly. I can’t imagine dealing with any language larger than C without a smart editor. It can be done but why?
I still find async annoying though.
My biggest source of friction with Rust (other than async) is figuring out how to write code that is both very high performance and modular. I end up using a lot of generics and it gets verbose.
> This is factually impossible.
> For anything larger than (very) small programs, Rust requires an upfront design stage, due to ownership, that it's not required when developing in GC'ed languages.
It seems that is not factually impossible.
Now, answering your question: It could be useful to use boxed types and later optimize it, so you get the benefits of rust (memory safety, zero cost abstractions) later, without getting the problems upfront when prototyping.
No it isn’t Both languages comprise much more than their memory management handling. Even if your premise is true, the conclusion does not follow.
Factually it's not. It may be true that in a very very idealized thought-experiment, when someone has a perfect knowledge, never makes mistakes, doesn't have preferences, can type arbitrary fast etc, python needs fewer keystrokes, fewer keywords or such, thus is faster. Who knows. But in reality none of the assumptions above hold, also literally everything plays a much bigger role in development speed anyway.
It's nearly the opposite. For larger programs in Python, you need an upfront design stage because the lack of static typing will allow you to organically accrete classes whose job overlap but interfaces differ.
Meanwhile, Rust will smack you over the head until your interfaces (traits) are well-organized, before the program grows enough for this to become a problem (or until you give up and start over).
How do I know? I'm stuck with some larger Python programs that became a mess of similar-but-not-interchangeable classes. RiiR, if I ever have the time.
You can also install pre-commit and mypy, and have static typing.
Definitely not - mypy's pretty good these days, and lots of people use it.
> But Rust's type system goes beyond just data types - lifetimes are also a part of the type system. I don't know how you can tack that on to Python.
Well, Python's objects are generally garbage collected rather than explicitly destroyed, so I don't think it'd make sense to have lifetimes? They don't seem a correctness thing in the same way that types do.
File handles, iterators, mutex guards, database transaction handles, session types, scoped threads, anything where ordering or mutual exclusivity matters.
I strongly agree from my own and my peers experience with the sentiment that latency from zero to running code is just higher in Rust than Python or Go. Obviously there are smart people around and they can compensate a lot with experience.
But nothing that Rust does slows me down because I’m used to it.
Having a robust type system actually makes refactors a lot easier, so I have less up-front planning with Rust. My personal projects tend to creep up in scope over time, especially since I'm almost always doing something new in a domain I've not worked in. Whenever I've decided to change or redo a core design decision in Python or Go, it has always been a massive pain and usually "ends" with me finding edge cases in runtime crashes days or weeks later. When I've changed my mind in Rust it has, generally, ended once I get it building, and a few times I've had simple crashes from not dropping RefCell Refs.
If you are only writing small programs where perf doesn't matter, crashing doesn't matter, design doesn't matter - Python will be faster than Rust, because the only thing that matters is how you can write the code from 0 to "done". You can jump right in, the design stage is superfluous & unnecessary. There is nothing to optimize, the default is good enough.
If you are doing slightly more than that, Python and Rust become about even, and the more you need those things, the better Rust becomes.
Now I'm wondering about demographics. Are people who love Python more likely to love Rust as well?
The rust-analyzer language server can autocomplete the available methods for a value.
>I know rust gives memory safety and how important that is, but the ergonomic is really bad. Every time I write some rust I feel limited. I always have to search libraries and how to do things. I cannot just "type the code".
You don't have to search libraries and figure out how to do things in Zig?
For me, it shares the most important benefits of Rust but with quite a lot more ergonomic coding model.
But me, I prefer to manipulate registers directly. Especially with "exotic" MCU where you have to execute write/read in specific number of CPU cycles. Rust makes that very hard.
Rust forces you to figure out ahead of time where each bit or byte is going to go and on which thread and using which mutation scheme. I’m happy to play the game, but it feels tedious for anything short of a parser or a microcontroller.
It messes with my process because I like to get something working before I determine the best API structure for it
I can get 90% of the performance with Swift and it flows much more easily, even though Rust’s type system is more powerful.
You can just clone and be on your merry way; you don’t need to worry about perf-related things if you don’t want to.
Swift makes every type implicitly copyable on the stack, including object pointers (through ARC), so you don’t have to clone. You can even pass functions as variables without any hoops.
I love lots of things about Rust, though, and will continue to use it for lots of things. Cross-platform things, for one!
Would add Go to the list, but only when I really have to.
Nim and Crystal could be alternatives, but don't seem to have big enough communities, at least for what I do.
However I do agree with the conclusion, Rust is a great language for scenarios where no form of automatic memory management is allowed, kernels, specific kinds of drivers, GPGPU programming, as general purpose, there are more productive alternatives, equally safe.
1) Modern Java is almost as good as C# with some things I can't give up in Java (static imports => succint code, Groovy Spock => succint tests)
2) Kotlin is better than C#
3) JVM has much much bigger ecosystem (almost all the apache projects are JVM oriented) and default web framework is much less code to type (SpringBoot) is much more productiv
4) JVM has wider variety of langs
For those reasons IMHO if you are small-mid company (or startup) it's wiser to choose JVM.
Also Java lost the attention span of the gaming industry, besides Android casual games and Minecraft, there are hardly anyone else paying attention to it.
Want a fastest possible library? It's in C++, and not portable to Win/Mac. So good luck with wrap + porting it.
Want a decent implementation of an algo? It usually exists for Java but not for C#. Hope you like writing it from scratch.
Want a C# implementation of an algo that doesn't allocate to the Nth degree. Again, write it yourself.
But ok, maybe Unity has a good ecosystem... And they fucked it over a barrel.
I also like Java, but c'mon no decent algorithms being implemented in C#? That is already approaching zealotry.
In C# the solution is half baked or archived or abuses allocation.
I think Unity has way more with C# adoption in game dev than innate C# qualities.
Sure you could use YAML parser, but it allocates everyone and their mother. Can you find a Fluent localization in C#? Sure, but its outdated and archived. Ok, but basic RoaringBitmap implementation? The repo is archived and not fully complete.
Why C# is used in game dev is incidental. It has more to do with Unity and XNA/FNA than any concrete quality of language modulo value types (but then again, most C# libraries don't focus on avoiding allocation and are just as happy as Java to construct a complicated hierarchy of classes).
Static import are also available in C# for quite some time now (c# 6, released in 2015, and in C# 10 you can even make this import global for for project).
I haven't used Kotlin, is there any killer feature compared to C#? (except more succinct code in certain cases?)
Also since it's Jetbrains - IDE integration is superior compared to anything C# can have (including Rider...)
Java doesn’t support C interop. For many desktop and embedded projects this is a showstopper, here’s an example https://github.com/Const-me/Vrmac/tree/master/VrmacVideo That C# code directly consumes V4L2 and ASIO Linux kernel APIs, and calls unmanaged user-mode DLLs like libfdk-aac.so and liba52-0.7.4.so.
Native stack and value types in C# reduce load on GC, and the number of complicated tricks required from JIT compiler. This in turn helps with startup performance. This is critical for command-line apps, and very desirable for desktop apps.
Another thing missing in Java is intrinsics support, both scalar like popcnt, bitscan, BMI, etc., and SIMD like SSE and AVX.
> Java doesn’t support C interop. For many desktop and embedded projects this is a showstopper, here’s an example https://github.com/Const-me/Vrmac/tree/master/VrmacVideo That C# code directly consumes V4L2 and ASIO Linux kernel APIs, and calls unmanaged user-mode DLLs like libfdk-aac.so and liba52-0.7.4.so.
Part of Panama: check out the "Foreign Function & Memory API" [0]. The official docs [1] say it is a preview in 21 but it got stabilized in Java 22 (isn't out yet).
> Another thing missing in Java is intrinsics support, both scalar like popcnt, bitscan, BMI, etc., and SIMD like SSE and AVX.
Also part of Panama: see the "Vector API" JEP [2].
> Native stack and value types in C# reduce load on GC, and the number of complicated tricks required from JIT compiler. This in turn helps with startup performance. This is critical for command-line apps, and very desirable for desktop apps.
This is part of Project Valhalla [3], they're adding value types and actual generics, among other things.
That said, most of these are not done / not in a stable LTS Java release yet. We'll see how much better it'll be compared to C# (if at all) once they land.
[0] https://openjdk.org/jeps/454
[1] https://docs.oracle.com/en/java/javase/21/core/foreign-funct...
Most real-live C APIs are using function pointers and/or complicated data structures. Here’s couple real-life examples defined by Linux kernel developers who made V4L2 API: [0], [1] The first of them contains a union in C version, i.e. different structures are at the same memory addresses. Note C# delivers the level of usability similar to C or C++: we simply define structures, and access these fields. Not sure this is gonna be easy in Java even after all these proposals arrive.
For a managed runtime, unmanaged interop is a huge feature which affects all levels of the stack: type system in the language for value types, GC to be able to temporarily pin objects passed to native code (making copies is prohibitively slow for use cases like video processing), code generator to convert managed delegates to C function pointers and vice versa, error handling to automatically convert between exceptions and integer status codes at the API boundary, and more. Gonna be very hard to add into the existing language like Java.
> "Vector API" JEP
That API is not good. They don’t expose hardware instructions, instead they have invented some platform-agnostic API and implemented graceful degradation.
This means the applicability is likely to be limited to pure vertical operations processing FP32 or FP64 numbers. The rest of the SIMD instructions are too different between architectures. A simple example in C++ is [2], see [3] for the context. That example is trivial to port to modern C#, but impossible to port to Java even after the proposed changes. The key part of the implementation is psadbw instruction, which is very specific to SSE2/AVX2 and these vector APIs don’t have an equivalent. Apart from reduction, other problematic operations are shuffles, saturating integer math, and some memory access patterns (gathers in AVX2, transposed loads/stores on NEON).
> most of these are not done / not in a stable LTS Java release yet
BTW, SIMD intrinsics arrived to C# in 2019 (.NET Core 3.0 released in 2019), and unmanaged interop support is available since the very first 1.0 version.
[0] https://github.com/Const-me/Vrmac/blob/master/VrmacVideo/Lin...
[1] https://github.com/Const-me/Vrmac/blob/master/VrmacVideo/Lin...
[2] https://gist.github.com/Const-me/3ade77faad47f0fbb0538965ae7...
My point is if you are doing business (especially web) apps. Use one of JVM langs insted of C# because ecosystem is much bigger (and it has fresher langs as well like Kotlin - if that's what you care about)
Despite having to spend extra time translating C API headers into C#, the productivity gains of the higher-level memory safe language were enormous.
Another example, I have shipped commercial embedded software running on ARM Linux, and based on .NET Core runtime. The major parts of the implementation were written in idiomatic memory-safe C#.
> doing business (especially web) apps
Well, these business web apps are precisely the long-running servers I have mentioned. Still, the software ecosystem is not limited to that class of problems, and due to different tradeoffs Java is not great for anything else.
#2 => don't know enough about Kotlin to comment
#3 => agree but quality > quantity
#4 => not terribly important to me (Clojure is cool but it's not "switch to the JVM" level cool)
My biggest issue with C# though is how badly exceptions are handled given that it is a statically typed langauge. I wish functions explicitly defined the exceptions it can throw since a minor package bump could add an exception without your compiler warning you that it isn't handled. I much prefer Rust, Go and Zig's error handling to C#'s since those kind of issues don't happen.
As clarification for the audience, it isn't the only GC enabled language with C and C++ like capabilities, in fact there are several examples since the early 1980's.
The adoption push for Java and scripting languages distorced the understanding of what was already available out there.
And those that have Java expertise but not C# seem to demur to those that do; image!
But it's still niche(around here and in the startup world) and gets lumped in with Java and together they are not "hip" or "agile" or whatever.
The only issue I have is with the .NET ... that is, building self-contained binaries to distribute.
For comparison:
* Hello World win-x64 binary self-contained in .NET 7 is around 70 MB
* The same for Go results in 1.2 MB
Edit: Missed 'trimming' in .NET, which would result in a binary of size around 11 MB in win-x64
Also remember that standard .NET runtime does a little bit more than Go's runtime, so it might happen that even with trimming, for basic applications Go ends up having an upper hand on file size.
On the other hand, I have had Go static binaries grow up to 200 MB and being require to use UPX to make it manageable, e.g. trivy.
Also note that triming only works properly if the libraries have taken the effort to be trimmable, as the linker errs on the safe side and won't trim unless certain that it is really dead code, and not called via reflection.
dotnet publish -c release -p:PublishAot=true- There is no point in chasing smallest possible binary size if it trades off performance and features one wants to use in production scenario, or comes with other tradeoffs that sacrifice developer productivity. I don't see anyone complaining about the size of GraalVM native images. As long as binaries are reasonably sized, it's not an issue.
- dotnet publish -c release -p:PublishAot=true definitely produces smaller binaries than Go as of .NET 8 (and no, you cannot use the argument that it's not released yet - it's in RC.2 which is intended for evaluation for adopting .NET 8 scheduled for release next month)
I have built a Win32 desktop app with its core logic in Go; and then re-built from scratch using .NET (v7). The core logic involved a fairly complicated keyboard input processing based on bunch of config files.
- Final binary of .NET ~ 14 MB
- Final binary of Go ~ 2 MB
And I was upset that they were so big. Sometimes I used UPX. Or I kicked out all Delphi GUI libraries, and created the GUI with the Win32 API calls directly. I got 50 KB Hello Worlds.
15k May 3 2019 quickrun.exe*
Win32 GUI Application that spawns Window and ask for alias to run. Pure Win32 API, written in C (Mingw).
I literaly looled at 11MB hello world of .net or 1.2MB Go..
I do not have Windows 98 anymore. But I still have Delphi 4 installed under Wine, so I just tried it out.
Just showing a messagebox from windows gives 16k
Using the sysutils unit though, puts it at over 40k. And with classes, it becomes 57k. Not sure what they pull in. sysutils contains number/datetime parsing and formatting, and exception handling. classes has basic containers and object oriented file handling.
But what libraries are you lacking?
And type erasure isn't as negative as you seem to make it.
Bunch of really obscure use cases - fluent localization, roaring bitmaps and so on.
I've been out of the windows development game for a long time, so I haven't used C# since it strictly required a VM... what's pre-compiled C# development like nowadays? Are there major caveats? If you can emit plain old binaries in C# with no runtime dependencies, that would make it a truly compelling language IMO.
And as another question, what's the cross-platform (mainly Linux) support like in AOT-compiled C#? If it's just as good as in Windows and emits plain executables, I would probably consider it the best place to start for any new project. (Something tells me it's not...)
On Microsoft side the Singularity and Midori experiments used AOT.
They influenced the AOT toolchains for Windows 8 store apps with MDIL (Singularity/Bartok), and Windows 10 store apps with .NET Native (Midori/Project N).
Now there is Native AOT, which supports CLI and native libraries,.NET 8 extends that to EF and ASP.NET frameworks. For GUI applications, maybe only fully on .NET 9.
Mono AOT has had support for ages, being used on iOS, Android, and Blazor.
Finally there is IL2CPP and Burst compiler from Unity.
IMO a good GPR is memory safe (no C, C++, Zig), is easy to use(no Rust), has strong static typing (no Perl, Python, Ruby) and is "stable" (no Scala). Lots of choices remain: Java, Kotlin, Ada, D, OCaml..
Not sure if much has changed (it was a while back), but my biggest problem was with finding and using 3rd party libraries (mostly for the boring stuff like DB connectivity, JSON/YAML parsing, logging, etc.).
E.g. even now if I search for "zig mysql library", the tops hits are all about people discussing it on reddit, than any actual library.
You can always improve it, but there's no need to. With Rust, I was never happy, even after 5 years, I was still thinking about better abstractions and implementing more traits, making it more generic, etc.
Zig doesn't have any of that. It's very interesting combination of low-level, predictable code, with meta-programming, where you get some of that abstraction back.
i.e. Zig does not have generics, but your function can return type, so generic list is just a function which returns a newly created struct.
If a function is called in comptime, it can also return types. So for example:
// this is a function which accepts a type
// if you accept type, you also have to restrict the arg to be comptime
// if the arg is comptime it still does not mean that the function cannot be called in runtime,
// but in this case, it returns type, so it is comptime-only function
// there are also cases where this is not true, like std.mem.eql(u8, a, b) which accepts type,
// but can be called in runtime because it does not return type
fn Wrapper(comptime T: type) type {
return struct { value: T };
}
const F32Wrapper = Wrapper(f32);
// @TypeOf(x.value) == f32
var x: F32Wrapper = .{ value: 1.0 }And you just keep refactoring/improving/desperately trying different ideas, because it never feels right.
It's ok if you don't agree but pls don't try to make my responses look like I like Rust, I don't and I'd happily get those years back if it was possible.
Rust feels like totally missing the sweet spot for me. It's way too pedantic about low level stuff for writing higher level applications, but way too complicated for embedded or writing an OS. In the former case I would rather take a C++, Java, Haskell, OCaml or even Go, and maybe sprinkle some C, and in the latter case C in macroassembly mode is far more suitable.
I still have a feeling that original vision of Graydon Hoare (i.e. OCaml/SML with linear types, GC, stack allocations, green threads and CPS) would be a much better language.
We as a profession have proven that we can’t write unsafe code at scale and avoid these problems. You might be able to in hand whittled code you write but what happens when other people work on it, it gets refactored, someone pulls in a merge without looking too closely, etc., or even maybe you come back two years later to fix something and have forgotten the details.
Having the compiler detect almost all memory errors is necessary. Either that or the language has to avoid this entirely. Rust is the former class unless you use unsafe, and the fact that it’s called “unsafe” makes it trivial to search for. You can automatically flag commits with unsafe in them for extra review or even prohibit it.
Unless the "poor ergonomics" and lack of shortcuts are explicitly what provides the static memory safety.
Apart from the static memory safety USP, Rust is repeating too many problems of C++ for my taste, and at a much faster pace.
The problem is that there is too much syntax, too many symbols, and too many keywords. I just keep forgetting how to use impl and lifetimes and single quotes and whatnot. It makes it really tough to use as an occasional language. And if I can't do that, then how can I get confident enough to use it in my job?
https://aardappel.github.io/lobster/memory_management.html
It uses compile-time reference counting / lifetime analysis / borrow checking. Which is mostly inlined to the point that there is none of the sort in the compiled output and objects can even live on the stack. It basically looks like Python but is nothing like it underneath, and of course with no GIL. You can run it on JIT or compile it to C++.
There's also Koka with the Perceus algorithm on compile time and looks like a much cleaner language than Rust. It also tracks side effects of every function in a type where pure and effectful computations are distinguished.
This is exactly how I feel about Rust.
There are some good ideas in there, hiding behind a horrible language design. I am waiting for someone to provide a more developer friendly alternative with the simplicity of C or Go.
FWIW there is interest in adding bounds checking to C [1]. That discussion includes feedback from at least one member of the C standards committee.
[1] https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-...
https://devblogs.microsoft.com/cppblog/build-reliable-and-se...
Rust will reject a lot of sound programs, and that's a huge performance hit. You are hitting the incompleteness wall with multiple mutable borrowing, closures in structures are a huge source of pain as well. And usually the answer from the community is "use handlers instead of pointers", but this gives you, surprise surprise, manual resource management alike that in C, and the compiler won't help much.
Of course this is all subjective but for me the ergonomics of Rust is far too bad. It's a good step in right direction along with ATS, but I really hope we could do better than this.
Having used Rust for a long time it’s definitely the biggest source of confusion and headaches.
> I always have to search libraries and how to do things.
Once you pass the initial curve with crutch like Copilot, then you can be almost as productive (if not more, considering refactoring and testing) with your native first coding language
Maybe it was a reaction against the Java-style reverse DNS notation, which is verbose and annoying, but a more GitHub-style user/group namespace prefixing package names would have been the nice middle ground.
Also, even if you know it's the latter, package namespacing isn't strictly related to directory structure, so `bar/baz` has no specific meaning outside of the context of a go import. They could have used any other separator for package components - `git.example.com/foo:bar:baz` - but instead they chose the slash, making the scheme both technically ambiguous and easy to confuse for an HTTP URL.
I still don't think this is a huge issue, to be honest. Not one big enough for me to complain about, for sure. But it's definitely not ideal.
Incorrect. You're able to use any URL you control, regardless of where your SCM is located.
By the time you're going to production, your vetted and locked dependency should be living in your own cache/mirror/vendored-repo/whatever so that you know exactly what code you built your project around and know exactly what the availability will be when you build/instantiate your project.
Your project shouldn't need to care whether GitHub fell out of fashion and the project moved to GitLab, and definitely shouldn't be relying on GitHub being available when you need to build, test, deploy, or scale. That's a completely unnecessary failure point for you to introduce.
Systems that use URL-identified packages can work around some of this, but just reinforce terrible habits.
E.g. doi's https://en.m.wikipedia.org/wiki/Digital_object_identifier
Basically the URL of a package name should be primarily the ID, not the locator (even if it is used for location by default).
In the Go world this would be "vendored" dependencies, that is, the dependencies are within your source tree, and your CICD can build to its hearts content with no care in the world about the internet because it has the deps.
The URL is useful for determining which version of a specific project is being used - "Oh we switched to the one hosted on gitlab because the github one went stale"
The advantage of using gitlab, or github, or whatever public code repository is that you get to piggy back off their naming policies which ensure uniqueness.
But, at the same time, there's no reason that the repo being referred to cannot be in house (bob.local) or private.
Having said all of that, the Go module system is a massive improvement on what they did have originally (nothing) and the 3rd party attempts to solve the problem (dep, glide, and the prototype for modules, vgo), but it's not without its edge cases.
I.e., when you're going to run the production build, the URLs are mapped to fetch from the vetted cache and not the internet.
I don't see any downsides to allowing them as a source, or making them the default approach
This seems strange to me because the whole point of a Uniform Resource Locator is to specify where a resource can be located.
It's a bit like saying "My project depends on the binder on shelf 7 in Room 42, sixth binder from the left. Except when I go into production, then use...." Don't tell me what binder it's in, tell me what it is.
I can see a case made for URIs, which is basically what Java did.
[dependencies]
uniffi = { git = "https://github.com/mozilla/uniffi-rs" }
Alternatively, you can do:
[registries]
maven = { index = "https://rust.maven.org/git/index" }
[dependencies]
some-package = { index = "maven", version = "1.1" }
In theory, all you need to do is publish a crate is to `git push upstream master`, and your package will become available on https://github.com/username/crate-name (or example.com/your-package if you choose to host your git repo on there).
Personally, I don't like using other people's URL packages, because your website can disappear any moment for any reason. Maybe you decide to call it quits, maybe you get hit by a car, whatever the reason, my build is broken all of the sudden. The probability of crates.io going down is a lot lower than the probability of packages-of-some-random-guy-in-nebraska.ddns.net disappearing
I suspect it was less a reaction against anything and more just following the norms established by most other package managers. NPM, PyPI, RubyGems, Elixir's Hex, Haskell's Cabal... I'm having a hard time thinking of a non-Java package manager that was around at the time Rust came out that didn't have a single, global namespace. Some have tried to fix that since then, but it was just the way package managers worked in 2014/2015.
"We're pretending security is not an issue." has been the feedback every time this is raised with the Cargo team.
To be honest, it's turned me off Rust a little bit.
The attitude of "Rust is memory-safe, so we don't need any other form of security." is not a good one.
Do you have a specific link where I can read this response, because this is not at all the responses I have read.
- No strong link between the repo and the published code.
- Many crates were spammed that were just a wrapper around a popular C/C++ library. There's no indication of this, so... "surprise!"... your compiled app is now more unsafe C/C++ code than Rust.
- Extensive name squatting, to the point that virtual no library uses the obvious name, because someone else got to it first. The aformentioned C/C++ libraries were easy to spit out, so they often grabbed the name before a Rust rewrite could be completed and published. So you now go to Cargo to find a Rust library for 'X' and you instead have to use 'X-rs' because... ha-ha, it's actually a C/C++ package manager with some Rust crates in there also.
- Transitive dependencies aren't shown in the web page.
- No enforcement or indication of safe/unsafe libs, nostd, etc...
- No requirement for MFA, which was a successful attack vector on multiple package managers in the past.
DISCLAIMER: Some of the above may have been resolved since I last looked. Other package managers also do these things (but that's not a good thing).
In my opinion, any package manager that just lets any random person upload "whatever" is outright dangerous and useless to the larger ecosystem of developers in a hurry who don't have the time to vet every single transitive dependency every month.
Package managers need to grow up and start requiring a direct reference to a specific Git commit -- that they store themselves -- and compile from scratch with an instrumented compiler that spits out metadata such as "connects to the Internet, did you know?" or "is actually 99% C++ code, by the way".
Maybe the obvious names should have been pre banned. But I don't see the issue with non-obvious names either way you're going to have to get community recommendation/popularity to determine if
brandonq/xml is better or worse then parsers/xml
You don't want to be using Ivan Vladimir's OAUTH package to sign in to Microsoft Entra ID. That probably has an FSB backdoor ready to activate. Why use that, when there's an equivalent Microsoft package?
When any random Chinese, Russian, or Israeli national can publish "microsoftauth", you just know that some idiot will use it. That idiot may be me. Or a coworker. Or a transitive dependency. Or a transitive dependency introduced after a critical update to the immediate dependency. Or an external EXE tool deployed as a part of a third-party ISV binary product. Or...
Make the only path lead to the pit of success, because the alternative is to let people wander around and fall into the pit of failure.
I think we need to encourage a culture that package managers are our shared garden and we must all help in the weeding.
> Do you have a specific link where I can read this response, because this is not at all the responses I have read.
Those aren't people saying security isn't an issue but examples of concerns you have which is different.
For some of those, there are reasonable improvements that can be made but will take someone having the time to do so. While the crates.io team might not be working on those specific features, I do know they are prioritizing some security related work. For instance, they recently added scoped tokens.
For some, there are trade offs to be discussed and figured out.
Literally nobody has said this.
> The attitude of "Rust is memory-safe, so we don't need any other form of security." is not a good one.
Fortunately it's an attitude that nobody in the Rust project has!
I know of a few people, personally, who have said this.
Because it seems like the people who are working on the project aren’t saying that.
The point is that it's much easier to make a mistake typing "requests" than " org.kennethreitz:requests" (as a pure hypothetical.)
It also means that more than one project can have a module called "utils" or "common", which once again reduces the risk of people accidentally downloading the wrong thing.
Sorry what? It's strictly the opposite: more character to type equals more risks to make a mistake.
In fact, in the general case, namespace increase the risk of supply chain attacks, because it makes packages names even less discernable.
jiggawatts
The implication here is that namespaces in package managers weren't a known concept. Outside Java, NPM - probably the biggest at the time - not only supported them but was actively encouraging them due to collective regret around going single-global in the beginning. Composer is another popular example that actually enforced them.
Not only was namespacing a known widespread option, with well documented benefits, it was one that was enthusiastically argued for within the Rust community, and rejected.
> The most prominent feature driving the release of npm 2 didn’t actually need to be in a new major version at all: scoped packages. npm Enterprise is built around them, and they’ll also play a major role when private modules come to the public npm registry.
My memory is that the industry as a whole didn't really start paying attention to the risks posed by dependencies in package managers until the left pad incident.
To be clear, I'm not saying that it was a good idea to not have a better namespace system or that they were completely ignorant of better options, just that they were very much following the norms at the time.
That crates.io launched without explicitly acknowledging this whole problem is either naivety or worse: already by then Java wasn't "cool" and the "cool kids" were not paying attention to what happened over there.
It's not that the industry wasn't paying attention until the 'left pad incident' -- that only holds if one's definition of "the industry" is "full stack developers" under the age of 30; I remember when that happened and I was working in a shop full of Java devs and we all laughed at it...
Maven's biggest problem was being caked in XML. In other respects it was very well thought out. That and it arrived at the tail-end of the period in which Java was "cool" to work in.
Rust chose to follow the majority of languages at the time. Again, as I noted in my previous comment, I'm not defending that decision, just pointing out that most of the widely-used languages in 2014 had a similar setup with similar weaknesses.
Also npm's bad policy/decision to transfer control of package in the name of predictability(this should probably be avoided for packages that aren't malicious. You could argue for seizing broken/trivial and unmaintained packages that have a good name but even then it might be best to leave well enough alone).
I suppose you're talking about the original dispute which led the developer to unpublish his libraries (which npm stupidly allowed, and cargo didn't). There's a smaller chance of a company wanting a random package namespace then a package name but its not impossible (think Mike Rowe Soft vs Microsoft)
It was "about" cavalier approach to the dependency supply chain. A dependency disappearing outright is just one of many failure modes it has.
This may be a little off topic for this comment thread but this is a little misrepresentative. Hosted private repos for enterprise weren't exclusive to Java at the time of left pad - anyone doing enterprise node likely had one for npm too & were probably well prepared for the attack. Such enterprise setups are expensive though (or at least take a level of internal resources many companies don't invest in) leaving the vast majority exposed to both js & java supply chain attacks even today.
[0] https://github.com/composer/packagist/issues/163#issuecommen...
Technically not in the same category, but Docker Hub (2014) had namespaces.
They were trying to do better than mainstream languages in other areas and succeeded. IIRC on this front they just decided Ruby's bundler was the bee's knees.
And at that time, it was a good idea. Ruby was popular and bundler + gem ecosystem was a big reason for its popularity. No one was worried that Rust might become so popular that it might outgrow the bundler model. That was only a remote possibility to begin with.
1. Flexible version (of requirements) specification;
2. Yes, source code had domain names in packages but that came from Java and you can technically separate that in the dependency declaration;
3. You can run local repos, which is useful for corporate environments so you can deploy your own internal packages; and
4. Source could be included or not, as desired.
Yes, it was XML and verbose. Gradle basically fixed that if you really cared (personally, I didn't).
Later comes along Go. No dependency management at the start. There ended up being two ways of specifying dependencies. At least one included putting github.io/username/package into your code. That username changes and all your code has to change. Awful design.
At least domains forced basically agreed upon namespacing.
"github.io/username/package" is using a domain name, just like Java. Changing the username part is like changing the domain name--I don't see how this is any worse in Go than in Java.
If you don't like that there's a username in there, then don't put one in there to begin with. Requiring a username has nothing to do with Go vs. Java, but rather is because the package's canonical location is hosted on Github (which requires a username).
I don't know why so many programmer's use a domain they don't control as the official home of their projects--it seems silly to me (especially for bigger, more serious projects).
So that old version of package xyz will still resolve in your tagged build years from now even if the project rebrands/changes namespaces.
Not only that, but a commercial, for-profit domain that actively reads all the code stored on it to train an AI. Owned and run by one of the worst opponents of the OS community in the history of computing.
At least move to Gitlab if you must store your project on someone else's domain.
To be honest I really like how Java advocated for verbose namespaces. Library authors have this awful idea that their library is a special little snowflake that deserves the shortest name possible, like "http" or "math" (or "_"...).
This was solved for C programs since whenever makedepend came out! (I'm guessing the 80s.)
(Bonus grief: Maven's useIncrementalCompilation boolean setting does the opposite of what it says in the tin.)
Neither node nor maven ever bothered to solve this, so we end up wandering the Wild West wondering when it will be that HR, or legal, or architecture comes knocking on the door to ask what we were thinking having a dependency on a dynamic version of Left Pad.
I'd kinda like to see what Cloudflare and Let's Encrypt could come up with if they worked together on at very least a white paper and an MVP POC.
Pay somebody either internally or externally to maintain a repo of all your dependencies and point your code at that. You won't get a left-pad incident. You won't get a malicious .so incident (unless you mirror binaries instead of source code).
Like if you ran out of screws to make your product with do you walk around the street and scrounge up some? No, you go to a trusted vendor and buy the screws.
I'm suggesting that the guys who've repeatedly proven themselves technically competent around security at scale might have a couple of useful ideas regarding how the industry might go about crawling its way out of this little security clusterfuck.
And perhaps even stop treating something as simple as a BOM as an enterprise feature, given that the overhead on such things is damned near zilch and the security implications are staggering.
They're all checked into Google3 (or chromium, etc.). One version only. With internal maintainers responsible for bringing it in and multiple people vetting it, and clear ownership over its management. E.g. you don't just get to willy nilly depend on a new version -- if you want to upgrade what's there, you gotta put a ring on it. If you upgrade it, you're likely going to be upgrading it for everyone, and the build system will run through the dependent tests for them all, etc.
And the consequence is more responsible use of third party deps and less sprawling dependency trees and less complexity.
And additional less security concerns as the code is checked in, its license vetted, and build systems are hunting around on the Internet for artifacts.
I'm not a fan of those - if the engine's in the wrong place, then why the hell did you fucking put it there?
Don't hack a patch into place to stop people from holding it wrong; design it in such a way that it's impossible to hold wrong in the first place.
https://samsieber.tech/posts/2020/09/registry-structure-infl...
The first section says it discourages forking. I consider this to be bad. Nobody’s code should be more important purely because it squatted a better name.
The Identity section actually makes the case that flat registries make naming harder.
The section on Continuity is “we’ve tried nothing and we’re all out of ideas”. Make up an org name and grandfather all packages in the flat namespace into that special org. Also this is already a problem because packages in the flat namespace do get abandoned, then forked, and then we have the associated issues.
The section on Stability seems to take it as a given that crates.io should be the only registry. I don’t. It also seems to conflate cargo with rustc for the benefit of the argument.
The squatting section describes only anti-features and I don’t consider the author’s legitimate use cases to be legitimate reasons to squat.
I think the only legitimate problems that need addressing are the ergonomics of accessing namespaced packages throughout transient dependencies and backwards compatibility with non-namespaced code. But the fact that these are real problems does not, to me, make a flat namespace a “feature”. It’s just easier to implement.
It’s okay for it to be a mistake that takes effort and time to fix.
You get some oddities such as serde-derive/serde-derive but the package owners can choose if they want to move to serde/derive or leave it in a separate namespace.
I send the analysis to the crates.io team and pointed that they have a no-automation policy.
They told me that it was not sufficient proof that someone was squatting those names. That's my problem with crates.io is that they have a clear policy and they don't enforce it so all the short/easy to remember names for crates are already taken and there is nothing you can do to get it.
Not naming names, but I know several people working to put Crates.io out to pasture.
There's a level of playing nice with them for the time being (eg. build reproducibility), but it's only KTLO.
Crates.io needs to die for Rust to thrive. They're a bungled, mismanaged liability. New code, new leadership.
pub use std::sic::semper; const step_on_snek = false;Once a better alternative is out there, crates.io will either wither and die or improve. If it matches its competition in terms of quality and reliability, everyone is better off. If not, the alternative solution will take over.
I'm eager for this crates.io alternative to land, assuming they don't break too many projects in their improvements.
******* is my password, but you can't see it. Type your password back, and I won't be able to see it. Try it!
hunter2
use::bar; // changing to
use::foo::bar;
I assume the library names that can be overridden in cargo would still be accepted, and then it all gets a little messy. The transition would be very messy.
Still doesn't solve all of the policy problems with namespacing.
What's that? I have scripts that automate publishing of new release of my crates. And I think many projects have.
What they stated was only regarding claiming new ownership over crate names:
> Using an automated tool to claim ownership of a large number of package names is not permitted.
> Using an automated tool to claim ownership of a large number of package names is not permitted.
And
- Hey, I found that someone created crates at a rate of about one every 30 seconds for a period of a week straight.
- That's not sufficient proof of squatting.
Whoever answered that, was either supporting the squatter or explicitly in favor of the practice. I cannot conceive that someone would get that evidence in their hands, and in their right mind think that the claim is bogus. Hell, I'd even be willing to suppress the squatter with evidence of one new crate created every 30 seconds for one hour!
The only reasonable conclusion to make is that they didn't really care. But then don't save face and claim that you do. That's hypocrisy.
So many inferior dependency management systems for other languages have come along later, and learned nothing from those that came before it.
The inverse-style domain name thing does a really good job of removing the whole issue of squatting from the ecosystem. You have to demonstrate some level of commitment and identity through control of a domain name in order to publish.
I would also say that this puts just enough friction so that people don't publish dogshit.
crates.io demonstrates quite clearly that you either have to go all the way and take responsibility for curation or you have to get completely out of the way. There is no inbetween.
It took some iterations before maven 3 became "good", so people forget that it wasn't as nice before now! Unfortunately, it seems that the lessons learned there is never really disseminated to other ecosystems - perhaps due to prejudice against "enterprisey" java. Yet, these package managers are now facing the sorts of problems solved in java.
Whats the diff between changing lib version in xml and json?
Maven files have a simple conventional structure and a defined schema. Once you learn it, it's a breeze to work with. It's not like you need to write hundreds of lines of SOAP or XLST — which is actually why people started to dislike XML, and not because XML inherently bad.
Edit: I'd also take XML over TOML any day, especially when it comes to nested objects and arrays.
Go has namespacing mostly because for a long time it didn't have a package manager at all, so people just used a bunch of ad hoc URL-based solutions mostly revolving around GitHub, which happens to have namespaces and also happened to lend itself to aliasing (because a whole GitHub URL is too long).
If you want to look at an actual example of namespacing done well, Maven/Java is the place to look. There is no aliasing—the same imports always work across projects.
It's definitely a good thing that people choose new unique names for crates rather than dijan/base64 vs dljan/base64
Do understand the desire of having a crate for audio manipulation called "audio" but at the same time how often do we end up with "audio2" anyway? It's an imperfect solution for an imperfect world and I personally think the crates team got it right on this one.
It's really as simple as this: many libraries are generic enough implementing something that already exists. Let's say you want a library to manage the SMTP protocol. On crates.io, of course someone has already taken the "smtp" crate (ironically, this one is abandoned, but has the highest download counts, because it's the most obvious name). Let's say you disagree with the direction this smtp crate has gone, and you make your own. What do you call it?
Namespaces solve this problem. You'd instead of have user1/smtp and user2/smtp competing in feature sets. You can even be user3/smtp if you don't like the first two.
This is precisely what Java enables too. The standard library is in com.java.*; if you don't like how the standard library does something, you can make com.grimburger.smtp and do it yourself. If you choose to publish to the world, all the more power to you. It doesn't conflict with the standard library's smtp implementation.
Naming things really is one of the hardest problems. This crates thing is a special case of Zooko's Triangle: https://en.wikipedia.org/wiki/Zooko%27s_triangle
Crates.io names are human-meaningful and everyone sees the same names, but it's vulnerable to squatting, spamming, and Sybil attacks.
You could tie a name to a public key, like onion addresses do, but it's unwieldy for humans. (NB, nothing stops you from doing this inside of crates.io if you really wanted)
You could use pet names where "http-server" and "http-client" locally map to "hyper" and "reqwest", but nobody likes those, because they don't solve the bootstrap problem.
It's a problem with all repos because when you say "http-server should simply be the best server that everyone likes right now", you have to decide who is the authority of "best", and "everyone", and "now". Don't forget how much useless crap is left in the Python stdlib marked as "don't use this as of 2018, use the 3rd-party lib that does it way better."
So yeah... probably will be a problem forever. As a bit of fun here are some un-intuitive names, and my proposed improvements:
- Rename Apache to "http-server"
- Rename Nginx to "http-server-2"
- Rename Caddy to "http-server-2-golang"
- Rename libcurl to "http-client"
- Rename GTK+ to "linux-gui"
- Rename Qt to "linux-gui-2"
- Rename xfce4 to "linux-desktop-3"
Then you only need to remember which numbers are good and which numbers are bad! Like how IPv4 is the best, IPv6 is okay, but IPV5 isn't real, and HTTP 1.1 and 3 are great but 2 kinda sucked.
Very simple. If a company as big as Apple can have simple names like "WebKit", "CoreGraphics", and "CoreAudio" then surely a million hackers competing in a free marketplace can do the same thing.
Or it is just a placeholder from a squatter.
Perl's whole ecosystem is amazing compared to other languages. It's a shame nobody knows it.
As other mentioned though, a typosquatting is a much bigger problem and namespacing offers only a partial solution. (You can still create a lookalike organization name, both in npm and in Github.)
brandonq/xml vs parsers/xml with no clue if one is better then another.
Also possibly with some confusion over whether things with the same name are forks or not. May make it a little more difficult to Google. Why not just have brandon_xml vs xml-parser and have a community list of best and most popular libraries?
I guess the only issue is that some generic/obvious package names are bad packages. That can have been avoided if they banned/self-squated most of the names. I suppose if you use dns namespaces and actually tie it to ownership of the domain name it might make sense but that would also cause issues(what if you forgot to renew the domain?).
We could even have `rust.xyz` for crates that are decoupled from `std` but still maintained by rust core devs such as `regex`.
You will have this problem only once. After an initial research you settle on one and move on with life.
I don't get how having to vet a dependency is going to be more difficult than before. The process is 99% the same, you still have to do the research work initially in both cases.
This is true that with no namespace anyone can end up squatting a cool name, but with namespace you end up in an even worse place: no-one end up with the cool names, and it makes discoverability miserable, because instead of having people come up with unique names like serde, everybody just name their json serialization/parsing library json and user now needs to remember if they should use "dtolnay/json" or "google/json" (and remember not to use "json/json" because indeed namespace squatting is now a thing), and of course this makes it completely ungoogleable.
We've had the namespace discussion for hundreds of time in the various Rust town squares, and the main reason why we still don't have namespace is because it doesn't actually answer the problem it's supposed to address and if you dig a little bit you realize that it even makes them worse.
Having a centralized public and permissionless repository opens tons of tricky questions, but namespaces are a solution to none of them.
Do you really think nobody has ever been able to google a Go import path? Some of these arguments are ridiculous.
What's the problem with that? You will have to explicitly state intention which is always a good thing.
> and of course this makes it completely ungoogleable
You are aware that just pasting username/projectname gives you exactly what you are looking for in the several top search engines, correct?
> We've had the namespace discussion for hundreds of time in the various Rust town squares, and the main reason why we still don't have namespace is because it doesn't actually answer the problem it's supposed to address and if you dig a little bit you realize that it even makes them worse.
OK, if you say so. Is this worse state of things documented somewhere?
> I questioned my sanity every time I circled back around to the Clippy issue above. Surely, I was wrong. There must be a configuration I missed. I couldn’t believe it. I still can’t. Surely there must be a way to configure lints globally. I quadruple-checked when I wrote this to make sure I wasn’t delusional.
You create a .cargo/config.toml in the workspace root so it covers all your crates.
inside the file:
[build]
rustflags = ["-Wclippy::lint_name_to_warn", "-Dclippy::lint_name_to_deny"]
It's fine. It won't be the only language I'll be proficient in while being averse to it at the same time. But I heard so many people proclaiming their love for it that I expected to enjoy it, too.
Also, just generally, it tends to make even simple things pretty complex. I understand why and am not really objecting to that, but it does make using it a bit like running with lead shoes.
I expect that the latter part may get better as I use it more (but perhaps not -- there are other languages I'm fully competent in but dislike for similar reasons).
For what it is worth, I found pursuing Zig to be a breath of fresh air. It's a promising alternative in the same space as C (and also Cpp and Rust). Checkout Andrew Kelley's introduction to the language.
For me, the key to understanding the borrow checker was understanding the underlying memory model. Rust memory model is the same as that of C, with some extensions for abstractions like generics. The borrow checker rules seem arbitrary at first. But it's deeply correlated to this memory model. The real value of the borrow checker is when I trigger it unintentionally. Those are bugs that I made due to lapse in attention. What scares me is that another language like C or C++ might simply accept it and proceed.
Yet another pleasant side effect of Rust's strict type system and borrow checker is that they gently nudge you to properly structure your code. I can say for certain that Rust has improved my code design in all the languages that I use.
1. Use Rust's runtime safety checks, using Rc, Weak, RefCell, etc. It will be as ergonomic as GC'ed languages (no productivity loss). While this has a runtime performance penalty, it will still be mostly comparable to other languages. This works for most use cases.
2. If you need the last ounce of performance, just drop all automated safety checks and do it manually using unsafe. Even if you make a mistake, your debugging will be limited to the unsafe blocks. This approach isn't unusual in Rust.
3. Use either the standard library or something on crates.io that does what's given in 2. Rust's generics make it easy.
This resource deals with these topics in-depth: https://rust-unofficial.github.io/too-many-lists/
2. No, I don't need every last bit of performance. C++ performance is OK. Or any other sane language like Nim, Crystal, etc. I need async, though, which is yet another hell in Rust.
3. This absolutely can't be done with standard library. There are libraries like PetGraph, which solve this particular problem (by some very unobvious approaches and bits of unsafe code), but there are many problems like it which don't have any libraries for it yet.
I do have hard problems. I want a language which makes hard problems easy, and impossible problems hard. Rust is definitely _not_ such a language, and it makes me like 5 times less productive, and sucks all joy out of programming.
I am a former C and C++ programmer who lived calling into pthread almost every week for a decade. I use async rust everywhere now.
I don’t get the hate that async gets. In my opinion, everyone should be using async for everything. Including stuff that’s seemingly single threaded “simple” stuff.
1. Fix rhai::Engine so it is Send (if !Send is unintentional)
2. Use tokio::spawn_blocking or normal threads to run the rhai::Engine bits
3. Don't hold the rhai::Engine across an await point.
Which one depends on rhai Engine details and what you want to accomplish.
Doing a block on inside a new thread seems unlikely to do anything useful (unless there's some undisclosed detail that makes it reasonable).
I encourage you to ask about this in the rhai repo in a discussion or issue.
2. I am using a normal thread but when I make it async the compiler wants Send.
3. The await point is in code that uses the engine. I am not sure there is another good option, since I need to use an API that has several libraries all of which are async.
The block_on is to allow the tokio Runtime created in that thread to execute/poll it
So, thank you for your input, I will test out the suggestion that I mentioned above, and then maybe look into spawn_blocking if that doesn't work.
You can have multiple runtimes, but create them ahead of time, in synchronous main, and keep a Handle to them.
But you probably need: https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html
This is likely because you are using the multi-threaded scheduler, which requires futures to be Send, even if you’re running only a single thread. This is because Tokio is based on a “work stealing” runtime, so in “normal” operations, expects the futures themselves to be able to be shuffled around threads where necessary.
For you use case, try running the single threaded executor, additionally try the “local set” executors. These do not require Send as they are statically guaranteed to be confined to the thread they are spawned on. Block on will also work.
Out of curiosity though, how is the interaction with Rhai performed, are you passing around the engine, and executing the code at certain points where applicable? Do you just need certain results from it sometimes? Etc?
I think I have a lot of good advice to go try now. Thanks.
From a quick look at the source, it looks like it switches between using Rc/RefCells and Arc/RwLock.
[0] https://docs.rs/rhai/1.16.2/rhai/#enable-special-functionali...
For instance, you apparently cannot `await` in the main function without a decorator you import from, you guessed it: Tokio.
Once you stop fighting the fact that tokio = async rust for 99% of cases, things are quite smooth.
Does "use async rust everywhere now" imply "use tokio everywhere now?" Honest question.
This is handy for I/O that can be interleaved and cancelled. You can (ab)use it for other things like generators or various DIY multitasking operations. It can also be a state machine generator (e.g. AI of actors in a game).
But I think OP just meant async for typical networking and DB interfaces. And yes, this usually implies the Tokio dependency.
Similarly, I find tasks that use or reuse large buffers to avoid the performance hits from allocation, often benefit from old fashioned thread pools. Bump or shard allocators can make this work ok, but in the cases where you are cpu bound on tight loops of vectorizable operations, thread pools perform better. Async is a good tool, but there are contexts it isn't optimal for.
I learned the opposite, that programmers tend to make everything unnecessarly async when it's not needed causing more complexity and mental load.
Rust's error messages are one of a kind. There no other compiler which comes even close.
As a side note, i used latex lately, it's error messages are horrendous. What a nightmare to figure out what's wrong by inspecting the error.
Generally good, but man do I hate how any error in my async function causes every recursive call site to generate an error about how the Future is no longer Send and Sync. Literally an entire console scrollback of errors with the actual syntax error buried somewhere in the middle.
In this case, the first two errors are clear. The next 3 provide multiple, redundant context blocks. The upshot is that this simple example results in rustc printing 11 lines of useful error messages and 86 lines of useless messages. Add to that the fact that the useful error messages need not be at the top or bottom of the error list, they can be anywhere inside of it, depending on the declaration order.
We do spend a lot of time trying to silence errors that are irrelevant. We also get a lot of complaints when fixing a single error produces a wave of new errors that were hidden due to failures in earlier stages. It's a balancing act.
Also, specific errors are verbose in order to give people a fighting chance to fix their issue. An error that is too verbose is annoying. An error that is too terse will leave users in the cold as to what the problem was. It's yet another balancing act.
I filed https://github.com/rust-lang/rust/issues/117233 for you. Could I ask you why you didn't consider doing so when you first encountered this problem?
Edit: good news, from the next stable onwards the issue you saw no longer reproduces. You only get the parse errors with your case. https://play.rust-lang.org/?version=beta&mode=debug&edition=...
Thanks for looking into it. I have a filed a bug against Rust before, but that was a clear bug, not a poor error message. Remember, the only time a user sees something like this is when they are trying to do something else. The only reason I looked into it just now is because you explicitly asked.
My main gripe is that I still don't fully comprehend lifetimes and the compiler can't really help me every time, because it (understandably) errs on the side of caution.
I'm not sure any compiler, even one "designed for diagnostics," can gather the "necessary metadata" from inside my brain based on my original intentions. If the compiler could unambiguously interpret what I meant to write, it wouldn't have needed to fail with a compilation error in the first place. Whenever I see something like "perhaps" or "maybe" in a compilation error, the information just didn't exist anywhere the compiler could possibly get to, and it's just a suggestion based on common mistakes.
This is a common refrain in C++ testing: if it compiles then it's probably correct.
> Rust has accounted for so many errors that many common test cases become irrelevant
In practice, if you think this way I think it's a sign that you aren't testing the right things in those other languages. You should be testing business logic, not language stuff.
If you look at your test code and think "I would test this in JavaScript, but I don't need to do that in Rust" then just delete the test.
I’ve heard this in Haskell and in Rust. I’ve never heard it applied to C++…
std::ptr::null() - an actual null pointer, probably the zero address on your hardware, and this isn't even an unsafe function. On the other hand, you won't find many uses for a null pointer so, I mean, congrats on obtaining one and good luck with that.
std::ptr::null_mut() - a mutable null pointer, similarly unlikely to be of any use to you in safe Rust, but also not an unsafe thing to ask for.
But, these are pointers, so they're not values that say, a String could take, or a Vec<String> or whatever, only actual raw pointers can be null.
First, this is a controlled panic, not a segmentation fault. The language is ensuring that we don't access the null pointer, or an offset from the null pointer. Null pointer access can be exploitable in certain circumstances (eg, a firmware or kernel). Your use of "exception" suggests you're thinking about it in Java terms however, and Java is equivalent here.
Second, you can only encounter this in explicit circumstances, when you have an Option<T>. Wheras in languages with a null type, any variable can be null implicitly, regardless of it's type.
But a segmentation fault is also controlled
>Your use of "exception" suggests you're thinking about it in Java terms however, and Java is equivalent here.
No, I am thinking in Delphi terms. It is overspecialized to Windows userspace. Windows gives an Access Violation, and that can be caught, and Delphi throws it as exception
>Second, you can only encounter this in explicit circumstances, when you have an Option<T>. Wheras in languages with a null type, any variable can be null implicitly, regardless of it's type.
Delphi has both nullable types and null-safe types
A segmentation fault may not happen (which is to say, we may corrupt memory or worse) if the null pointer is mapped into memory (as in an embedded or kernel context) or if it's accessed at a large offset, which results in a pointer that's mapped into memory. This may be rotten luck or it may be exploited by an attacker.
> No, I am thinking in Delphi terms.
Fair enough, I don't know Delphi. I'll take what you're saying about it as read.
How do null pointer exceptions arise?
Inconsistent data: you have code paths that implicitly assume invariants. Trivial cases like “this field is not provided” and more complex ones like “these fields have a specific relationship”.
You can often move those assumptions into the data structure in any language.
Then your tests become a matter of generating and transforming data from a holistic perspective instead of micromanaging individual code paths.
Remember unlike most languages you'd be familiar with C++ has IFNDR, which has been jokingly referred to as "False positives for the question: Is this a C++ program?". A conforming C++ compiler is forbidden from telling you in some† unknown number of cases that it suspects what you've written is nonsense, it just has to press on and output... something. Is it a working executable? Could be. Or maybe it's exactly like a working executable except it explodes catastrophically on Fridays. No way to know.
† The ISO standard does identify these cases, but they're so vague that it's hard to pin down everything which is covered. My guess is that all or most non-trivial C++ software is actually IFNDR these days. Just say No to the entire language.
It's not quite that bad - a conforming C++ compiler is permitted to error out and not compile the program. It just doesn't have to.
Surely in the undecidable cases the compiler is allowed to produce a binary that errors cleanly at runtime if you did in fact violate the semantic constraint, and any sane implementor would do that. (Not that any sane person would ever write a C++ compiler...)
It's not merely about whether your comparisons are transitive, the type must exhibit a total ordering or your sort may do anything, including buffer overflow.
> Surely in the undecidable cases the compiler is allowed to produce a binary that errors cleanly at runtime if you did in fact violate the semantic constraint,
I don't think I know how to prove it, but I'm pretty sure it's going to be Undecidable at runtime too in many of these cases. Rice reduced these problems to Halting, which I'd guess means you end up potentially at runtime trying to decide if some arbitrary piece of code will halt eventually, and yeah, that's not helpful.
I've written about it before, but I should spell it out: The only working alternative is to reject programs when we aren't sure they meet our constraints. This means sometimes we reject a program that actually does meet the constraints but the compiler couldn't see it.
I believe this route is superior because the incentive becomes to make that "Should work but doesn't" set smaller so as to avoid annoying programmers, whereas the C++ incentive is to make the "Compiles but doesn't work" set larger since, hey, it compiles, and I see Rust's Non-Lexical Lifetimes and Polonius as evidence for this on one side, with C++ 20 Concepts and the growing number of IFNDR mentions in the ISO standard on the other side.
Sure. But there's no requirement for the compiler to be a dick about it, and hopefully most won't.
> I don't think I know how to prove it, but I'm pretty sure it's going to be Undecidable at runtime too in many of these cases. Rice reduced these problems to Halting, which I'd guess means you end up potentially at runtime trying to decide if some arbitrary piece of code will halt eventually, and yeah, that's not helpful.
At runtime can't you just run the code and let it halt or not? Having your program go into an infinite loop because the thing you implemented didn't meet the requirements is not unreasonable.
I'm sympathetic to the idea that there could be a problem in this space, but without a real example of a case where it's hard for a compiler to do something reasonable I'm not convinced.
> I've written about it before, but I should spell it out: The only working alternative is to reject programs when we aren't sure they meet our constraints. This means sometimes we reject a program that actually does meet the constraints but the compiler couldn't see it.
> I believe this route is superior because the incentive becomes to make that "Should work but doesn't" set smaller so as to avoid annoying programmers, whereas the C++ incentive is to make the "Compiles but doesn't work" set larger since, hey, it compiles, and I see Rust's Non-Lexical Lifetimes and Polonius as evidence for this on one side, with C++ 20 Concepts and the growing number of IFNDR mentions in the ISO standard on the other side.
Meh. I'm no fan of the C++ approach, but I'd still rather see C++ follow through on its strategy than half-assing it and becoming a watered-down copy of Rust. People who want Rust know where to find it.
#define _FOO
int main() {}
> In addition, some identifiers appearing as a token or preprocessing-token are reserved for use by C++ implementations and shall not be used otherwise; no diagnostic is required. [...] Each identifier that contains a double underscore __ or begins with an underscore followed by an uppercase letter is reserved to the implementation for any use.
Thus, the compiler theoretically has the liberty to emit whatever it wants for this program.
Neither GCC nor Clang produces a warning under -std=c++20 -Wall -Wextra (Clang only produces a -Wreserved-macro-identifier under -Weverything), and MSVC doesn't produce a warning under /std:c++20 /Wall.
In practice, most examples of ill-formed programs where compilers issue no warnings occur with discrepancies between different source files that are linked together; e.g., declaring a function as inline in one file but non-inline in another, or declaring a function with two different sets of default arguments in different files, or defining the same non-inline variable or function in different files, or defining the same inline function differently in different files.
In theory, sure. In practice, what does it do? We can look and see.[0][1][2] (I don't know of any compiler that emits garbage in the example you listed). For some reason, there's this sentiment that's arisen that treats undefined behavior as some sort of bugaboo that's capable of anything and everything (including summoning nasal demons).
Here's a question that I can't find an answer to. Is machine code well-defined (as in, can it contain undefined behavior)? If the answer is yes, then all Rust programs can also contain undefined behavior, because they eventually turn into machine code after all. If the answer is no, then that means once a C++ program is compiled into machine code, the executable is a well-defined program.
This whole nasal demon garbage was a good hyperbole at the time to explain that, yes, undefined behavior is bad. But it's been taken to this extreme where people will use arguments like this to try and convince others that if your program has undefined behavior, then it could summon a nasal demon, because who knows what could happen.
In reality, that won't happen. In reality, a signed integer overflow can result in a meaningless program, or a security vulnerability, or many other bad things, but it doesn't mean that the compiled machine code is all of a sudden inventing new instructions at random that the CPU will happily accept. It doesn't mean that your program will turn your OS into a pac-man game. It doesn't mean that it's impossible to find the root cause of the issue and remove the undefined behavior.
In practice, undefined behavior means that your program is broken, but you can look at the compiled program and trace exactly what it does. Computers are machines, they behave predictably (for the most part). They fetch an instruction, execute the instruction, and read/store results to memory. If you look at the instruction and memory (the cache stored on the specific core that the processor is using), you can reliably predict what the CPU will do after executing that instruction.
And yes, multithreading exacerbates the effects undefined behavior and makes it more difficult to debug. But you can debug the issue, even if it means you have to look at the machine code directly.
So while, yes, a compiler can emit garbage when it encounters the example program you gave, using that as an argument for why undefined behavior is bad is dumb. Because in reality, compilers will do something that makes sense, and if it doesn't, you can just look at what it produced. In all the examples you listed, I'm sure that if you created a small example illustrating those situations, the compiled program would do what you expect it to. (Like in the case of having different default args, it will probably just throw an error like this[4]).
Sorry for the rant, it just annoys me that we can't discuss undefined behavior without somebody making an argument that doesn't matter in virtually every case. Undefined behavior is bad! But there are good, real, common examples that show why it's bad! Use those instead instead of talking about how compilers are technically permitted to spew trash when they encounter a program that looks normal, because I'd be very surprised if any modern compiler exists that does spew trash for a normal looking program.
This talk by Chandler Carruth seems pretty good on explaining the nuance of undefined behavior[5].
[0]: https://godbolt.org/z/zv4GhPK5E
[1]: https://godbolt.org/z/6obn7134G
[2]: https://godbolt.org/z/33cKcx5xs
[3]: https://en.cppreference.com/w/cpp/language/ub
[4]: https://learn.microsoft.com/en-us/cpp/error-messages/compile...
This is a conscious choice that C++ made, and my belief is that once you make this choice you have an incentive to make it much worse over time e.g. in C++ 11, C++ 14, C++ 17, C++ 20, and now C++ 23. Specifically, you have an incentive to allow more and more dubious code under the same rule, since best case it works and worst case it's not your fault because if it doesn't work it was never actually a C++ program anyway...
Still, since you decided to talk about Undefined Behaviour, which is merely unconstrained misbehaviour at runtime, let's address that too.
For the concurrency case no, humans can't usefully reason about the behaviour of non-trivial programs which lack Sequential Consistency - which is extremely likely under Undefined Behaviour. It hurts your head to even think about it. Imagine watching an avant garde time travel movie, in which a dozen characters are portrayed by the same actor, the scenes aren't shown in any particular order and it's suggested that some of them might be dreams, or implanted memories. What actually happened? To who? And why? Maybe the screenwriters knew but you've no idea after watching their movie.
Today a huge proportion of software is processing arbitrary outside data, often as a service directly connected to the Internet. As a result, under Undefined Behaviour the unconstrained consequences may be dictated by hostile third parties. UB results in things like Remote Code Execution, and so yes, if they wanted to the people attacking your system certainly could turn it into a Pac man game.
We should strive to engineer correct software. That we're still here in 2023 with people arguing that it's basically fine if their software is not only incorrect, but their tools deliberately can't tell because that was easier is lunacy.
From the link I posted, from cpp reference, which gives a definition for what constitutes undefined behavior:
>> ill-formed, no diagnostic required - the program has semantic errors which may not be diagnosable in general case… The behavior is undefined if such program is executed
And as for the rest of your argument, you still haven’t answered the one question that matters. Is an executable file with machine code well-defined? If it is, then once you compile a C++ program, the generated binary is well-defined. And once again, all the superfluous arguments about what could happen are irrelevant when there’s no context provided. There’s lots of different types of undefined behavior. An integer overflow by itself does not make your program susceptible to remote code execution. It’s the fact that the integer overflow gets stored in a register that later gets used to write to invalid memory, and then that invalid memory has harmful code injected into it that gets executed.
We should strive to engineer correct software. I agree. It’s all this hand waving about how all undefined behavior is equal that irks me. Because it’s not true, as evidenced by the example listed above about the program that’s “technically” ill-formed C++, but in practice compiles to a harmless program. Engineering requires an understanding of what could go wrong and why. Blindly fretting about all undefined behavior is useless and doesn’t lead to any sort of productive debates.
Engineering is all about understanding tradeoffs. Which is one of the reasons the C++ spec does not define everything in the first place. The fact that in 2023 people don’t understand that all decisions in software come with a tradeoff is lunacy. And once again, I agree that undefined behavior is bad, but to pretend that the existence of IFNDR means the whole language is unusable is silly. People use C++ everyday, and have been for almost 40 years. I’d like more productive conversations on how rust protects the user from the undefined behavior that makes an impact, not about how it doesn’t have IFNDR because that’s irrelevant to the conversation entirely.
While you've insisted that's somehow the one thing which matters I don't agree at all. Are programmers getting paid to produce "any executable file with machine code" ? No. Their employer wanted specific executables which do something in particular.
And there aren't "different types of undefined behavior" there's just one Undefined Behaviour. Maybe you've confused it with unspecified behaviour ?
In the integer overflow case, because that's UB (in C++) it's very common for the result to be machine code which entirely elides parts which could only happen under overflow conditions. Because overflow is UB that elision didn't change the program meaning and yet it makes the code smaller so that's a win - it didn't mean anything before in this case and it still doesn't - but of course the effect may be very surprising to someone like you.
The excuse that "We've done it for decades therefore it can't be a bad idea" shouldn't pass the laugh test. Did you notice you can't buy new lead paint any more? Asbestos pads for ironing boards? Radium dial wristwatches? "That's a bad idea, we shouldn't do that any more" is normal and the reluctance from C++ programmers in particular shows that they're badly out of touch.
There is already a crate providing parallel iterators. You just rename the iter() call and that's it. I don't agree it should be implicit though.
But to your point, the convenient defaults are very different. Unsafe typecasts require a lot of ceremony and careful thought in Rust, and they have to follow more rules than in C. In particular, references are all effectively "restrict" in Rust, and it's really easy to screw that up when you do unsafe cast from raw pointers to safe references, which is a big incentive not to write code like that if you have a choice.
Whereas Rust is a standard with multiple implementations?
[1]: https://learn.microsoft.com/en-us/cpp/assembler/inline/inlin...
> Inline assembly is not supported on the ARM and x64 processors.
If anything I want more explicit control, alleviated with an even more expressive type system. Ideally rusts type system would just be a prolog variant imho.
https://en.wikipedia.org/wiki/Category:Compiler_optimization...
Otherwise we should trust compilers to know better.
Bash with types (especially floats), fewer edge cases, functions with explicit parameters, simple command line flags...
edit: Although I can see that if they hadn't made string splitting the default, it would have saved a lot of headaches.
I guess I could avoid floats using multiplication, but I would rather keep the code as simple as possible.
I know it’s still not 1.0. Then just get fish. At least would save some keystrokes.
The Kotlin type system is comparable to Rust without the borrow checker. It has non-null types, generics, etc.
It's not really a "product" per se but there's an old version for download and some lightweight docs here:
We've never worked out what to do with it. Ideas and feedback welcome. It's pretty nice to use, albeit you need to use IntelliJ to edit scripts if you want IDE features.
Rust is quite transparent in what it does, and is very conservative with compiler magic. The language doesn’t do heap allocations, doesn’t do reference counting, it doesn’t even have implicit numeric type conversion. It won’t implicitly copy types that did not ask to be implicitly copyable, and even that is only legal for types that can be copied with a simple shallow memcpy.
Rust uses zero-cost abstractions all over the place, which means it’s predictable what code they will compile to, and that will be typically something simple. Std types have well-known basic layout, so you know that e.g. iteration over a Vec is going to be a loop incrementing a pointer, and there’s no implicit parallelism.
Describing borrowing as “compiler knowing better than the programmer” is a weird way of looking at it. Borrowing is like type checking. You declare a type to be temporary, and try to use it as long-lived, you get an error. It’s the same as if you declare function to return struct Foo, but returned struct Bar instead. Yes, compiler “knows better than you”, because you just wrote a bug.
Borrowing still compiles to direct pointer usage without a GC (it’s literally guaranteed to be identical to a C pointer in C ABI structs and functions), and you can override lifetimes with unsafe if you think know better than the compiler.
A good C compiler does this when you turn on all the flags. I like languages/compilers that let you selectively disable the screaming and let you write bad code on purpose. Bad code that works but can be written fast is often better than perfect code that takes forever to write. Once you have a bad but working POC, you can make it less bad.
"It’s got no problem attracting new users, but it’s not resulting in dramatically improved libraries or tools. It’s resulting in one-off forks that handle specific use cases."
Age has nothing to do with that. Attracting core devs is hard, and putting lots of effort into making it attractive is necessary. On top of that, cultural conventions are set by the early adopters, and a lack of convention is often just as sinful as a bad existing convention.
Take Python for example. They took a lackadaisical approach to development and runtime environments, and as a result there's 50 competing ways to develop or run a Python program. Their most-used package repository, PyPI, has been a mess for years. Nobody builds on top of existing packages, names make as much sense as a random word generator, the ecosystem is rife with malware, you can't even search for a package on the command line, etc etc. None of that is the language's fault, it's the community and core team's fault for sitting on the sidelines rather than leading. Culture matters more than the tech it's centered around.
(I'm not trying to pick on them, I just know their problems better. C has been around for a half century and its community never really put together half the solutions more modern languages did)
Rust supports that. Just mark everything unsafe.
You can instead wrap everything in Arc<Mutex<T>> and .clone() liberally, though.
There will probably be weird behavior, though, because Rust optimizes based on assumptions about boxes and references. For example, if you have 3 raw pointers to some object live at once, and you give some library function a mutable reference made from one of those pointers, the compiler will optimize assuming it has exclusive access to that object, and it may make incorrect assumptions.
And all those let you do is drop lifetimes and wave goodbye to the borrow checker. You just have to be explicit about it.
(What I mean is, the borrow checker checks borrows, it checks references. In unsafe you can make a reference forget what it's referring to. Just change &'a T to &'static T, and then the borrow checker is doing nothing.)
I checked the six most recently published crates on crates.io (blablabla, nutp, tord, g2d, testpublishtesttest, hellochi, at the time of writing). Three of those (blablabla, testpublishtesttest, and hellochi) did some variation on printing `hello world`. g2d seems like an interesting graphics library. tord provides a data structure for transitive relations, which is also neat. No crate contained over 250 lines of code. Unsurprisingly, none of them contained unsafe code.
Elsewhere in this thread, it's been pointed out that as a consequence of crates.io having a global namespace, plus lax enforcement of an anti-squatting policy, there are a lot of namesquatting packages. Those presumably contain no unsafe code.
tokio contains unsafe code. rand contains unsafe code. regex contains unsafe code. time contains unsafe code. (method: a smattering of packages chosen from blessed.rs; result: every one that I checked except serde containing unsafe code; epistemic status: eh -- I grepped the codebases, ignoring things that were pretty clearly tests, but might have accidentally included some example code or something that's not part of the core library? Please let me know if I've misattributed unsafe usage to one of these projects, or if I've managed to select a biased sample!)
I'd certainly believe a straightforward reading of the claim "80% of crates have no unsafe code"...but that seems almost meaningless, given that a not-insignificant portion of crates contain basically no code at all? I'd be much more interested in a weighted percentage by downloads: I'd be wildly impressed if 80% of crate _downloads_ contained no unsafe code, and would be somewhat unsurprised if the number was well below 50% -- crates with more functionality would be more useful and therefore more download, but also more likely to use unsafe code, I'd imagine.
Edit: I just noticed crates.io has a most-downloaded list[0] -- I might end up running some numbers on top packages there tomorrow morning, for some more solid data.
Also, you’re replying to someone who I’m fairly sure is on one of the core Rust teams, if not closely involved, I’m somewhat more inclined to trust _them_ when they say 80% of libs don’t contain unsafe (given that it cleanly meshes with my own experience of Rust libraries).
I'm not too concerned with performance or safety (since my code hasn't been seen very often).
I use rust just because it has a lot of "libraries that help me develop".
While other language libraries have a poor search experience and ranking system, Rust has a great library search system.
I use it because I don't have to read blog posts like C/C++ or java to find libraries that help me develop, or wade through unnecessary libraries like C# or go, I can use the rankings, and it has a good documentation system.
As you've said, finding great libraries is easy, and so is adding/building them. I find this a nice change from e.g. Python (where there are so many way of different competing ways dealing with packaging/dependencies).
Also thanks to the error handling, sum types and the like, I don't have to worry so much that I've deal with all errors, handled all enum variants, etc., the compiler takes care of that.
At the language level, the big problem is back references. Sometimes you do need them, and the only safe way to do them at present involves reference counts in the forward direction and weak references in the back direction. Then you have to call .borrow() and .upgrade() too much. I'd love to see a static analysis solution to that.
(Rough outline of such a solution: Owning object belongs to Owner trait. Owned object belongs to Owned trait. Owned object has .owner() and .owner_mut() functions which retrieve references to the owner. Owners probably have to be pinned, so they can't move while a back-reference exists. If an Owner changes a reference to an Owned object, the back reference is automatically updated.
That's the easy part. Now figure out how to prove by static analysis that a specific use of this does not violate Rust's no-aliasing rules (N read-only, or 1 mutable). This looks do-able for the non-mutable case, because having a non-mutable reference to both owned and owner is OK. Mutability, though, is tough. Anybody thinking about this?)
Is it fair to say that Rust failed to supply a robust set of standard libraries comparable to other modern languages? Or was the language aimed at level geared towards implementing rather than providing libraries? If it's truly a systems language, then what library features are essential, and what are 'nice to have'?
But it’s a trade off because there are concrete upsides to a large standard library. I wrote about this more - Rust has a small standard library (and that’s ok) - https://blog.nindalf.com/posts/rust-stdlib/
What I would encourage the Rust community to focus on is adding security-sensitive functionality to the standard library. Not the big wad of complexity that is cryptography, but a standard PRNG/RNG/CSPRNG would fit.
Also, I do think it's a bit of a mistake to not at least define a relational database API. As it is we have sqlite, mysql, and postgres with similar but slightly different APIs.
I don’t think low level languages mean you need a small standard library and high level means you need a large one. You’re pattern matching between Python, Java, Go (all large, older) and now Rust (small and newer).
Rust wants to be high level as well, allowing people to solve all kinds of problems of varying complexity in different domains. And to some extent, Rust has succeeded in empowering folks to do this. Rust has found its way into “systems” like browsers (Chromium, Firefox) and Operating Systems (Windows, Linux) but also web software, infrastructure (firecracker), developer tools (ruff, turbopack) and other domains.
Like I said, the request for a larger standard library is a reasonable one. But I don’t think it’s going to happen, because Rust seems to be gaining traction in multiple domains without it. By the numbers it looks like it’s growing 10x every 3 years (https://lib.rs/stats). In such an environment, the folks at the wheel would double down on their approach rather than fundamentally rethinking it.
They often have better quality than what I find in "mature" packages in npm or PyPi.
Rust kind of has a perfectionist touch to it which makes it really hard to say "this public API is stable". So people stay at 0.. way longer than normal.
I'm fine with that.
The lack of namespacing and heavy namesquatting is rather what's annoying.
Out of curiosity, is there any existing language (including research languages) that can do this already?
You can do this in Rust at run time, with Rc, Weak, and .upgrade().unwrap(). If none of the .unwrap() calls fails, you did it right. If you can prove the unwrap calls can't fail, you don't need the run time checking. So the goal is to check at compile time something you can now check at run time at higher cost. That's what Rust is all about.
I complained about this here[1]. I'm told "Tokio won, it's not a problem anymore." Wrong. It's still a problem and it's a killer.
Tokio monoculture is a serious problem.
First, I have to wonder whether any of this can be legitimately elevated to the level of a "culture." Right now, according to GitHub "insights," tokio-rs has approximately 2.0 regular, every day contributors. And that's the most widely used Rust async runtime. Everything else is likely even more thin.
Second, Tokio has a lot of share because it was early and aggressive in actually delivering usable async documentation and code. I recall, years ago, reading Tokio documentation on how futures worked and grasping these concepts all the way from file descriptors one might epoll() in C, up to the Rust abstractions, and thinking "hell, I could write an executor from first principles based on this." Tokio earned the advocates it has, as oblivious to the real state of things as some of them might be.
I think the real problem is that async Rust is incomplete. As I said elsewhere, async Rust syntax is fine. The implications of async Rust exposed some papercuts in the language that have had to be dealt with since, but the core syntax is fine. I believe that can be attributed to the serious minded attention that the syntax received from people way, waaay up the cognition curve. The part that didn't get enough thought was async runtimes. In an ideal world, one would develop a Rust library that utilizes and/or implements asynchronous calls and transparently, flawlessly run on any correctly implemented runtime alongside any other number of libraries also utilizing and/or implementing async calls.
That is not the case, and the damage that's doing is severe. For every one person, such as myself, that will dare attempt to articulate this pain and, in the process, certainly revealing clear evidence of blatant ignorance, a thousand others just silently gave up.
I've at least thought-experimented with what it would take to write my own code agnostic enough that it could run on both e.g. tokio and e.g. monoio etc. and, well, it just can't happen. Even if you find neutral/unbundled implementations of locks, channels, utilities, to depend on instead, you end up stuck at: task spawning, and any kind of I/O. The former, to me, is a glaring absence from the language standard; async should not have gone out the door without support for it.
But there are other, non-syntax issues, such as synchronization primitives, IO events, etc. that are clearly underspecified. No maybes about it.
And so, for some or all of these reasons, right or wrong, various Rust libraries wed themselves to runtimes other than Tokio.
You can thunk around these things, but it's miserable, yielding subtle, inscrutable code that you will not understand in six months when you have to maintain it. And the real fun begins when you need to handle Err, and all the abstraction leaks around Send and threads ruin your day; more thunking, Boxing errors (omfg) and even subtler and more inscrutable code.
I wonder what kind of tests the author was writing? The test cases I write for my code is for weird edge cases I detect during testing, like parameter over or underflows, correctness of parsing functions, etc. Things that do fail. I don't get why you'd write tests for trivial things that can't fail.
I think it's a matter of perspective/background. Depending what language you're coming from you might appreciate a lot of the more modern language features, or binary size, or performance. On the other hand, you might already be working with a pretty cutting edge language with a garbage collector or other and decide that Rust has some cool benefits but not worth the switch.
Rust is great for services such as API endpoints, where low latency and high throughput at a low cost are more important that hot reload during development.
The experience has been far more than just borrow checking, it’s opened up my mind to a lot of concepts that my previous trials with Java, JavaScript, and Python fully obscured in arcane ways. Rust’s compiler is wonderful, and once you get past the ergonomic struggles (which I did by just going through Advent of Code), you’re set.
Even when you don't care about performance, another issue that comes up sometimes is keeping track of mutable state. If you've ever relied on bytes instead of bytearray or tuple instead of list* to guarantee that no mutation is happening in some Python code, you know what I'm talking about. Rust can give you a similar level of control over who gets to mutate what, without making you change the type of your data or pay the cost of copies. It's basically the const/non-const distinction from C, but much stricter. Another way of saying the same thing is that Rust gives you a lot of the legibility/correctness benefits that you'd expect from a functional programming language, but you get to code in the usual imperative style.
* And even then, that only prevents assignment to the elements of the tuple. You can still mutate an element internally if it's not also an immutable type.
As long as you don't do a couple of things that, while easy to remember not to do once you know them, a programmer used to languages like Python and Javascript might be oblivious to. Things like:
* Forgetting to use buffered IO wrappers
* Using println!() for high-volume console printing rather than locking stdout and writing to it manually
* Everything is a struct with concrete types. This is closest to C. It benefits from the improved memory safety, without sacrificing speed.
* Everything is a Box<Rc<T>>. This is closest to dynamic garbage-collected languages, but with vastly improved performance.
* Everything is an “impl Trait”. This is closest to templated C++, but with much better ergonomics.
So, while I think I agree with your specific statement that the improved memory management may only be fully appreciated by those that cut their teeth on C’s segfaults and dangling pointers, I don’t think that’s the only benefit of Rust as a language.
Wait, reference counting everything all the time is faster than normal garbage collection?
The performance difference, though, is mainly in switching over to a compiled language in the first place.
Why the `Box`?
If you've never used an ML family language then Rust might still be a breath of fresh air even if you don't care about memory management.
> It also looks like (soon) you’ll finally be able to configure global lints for a project. Until now, you had to hack your solution to keep lints consistent for projects. In Wick, we use a script to automatically update inline lint configurations for a few dozen crates.
It's trivial to configure global lints per crate. The author's problem comes down to having multiple crates per project. Which is something people do sometimes, and should be supported, but there's a big difference between the two interpretations. I would go so far as to say most Rust projects are only one crate (and so support "global" linter configuration)
Do you need bare metal performance? Then why use Rust when you could use C or C++, which have much larger ecosystems, platform support, more mature tooling, etc.
Do you need BOTH memory safety and baremetal performance at the same time? Then there really aren't many other options besides Rust.
But what I've started wondering lately is: are there really that many situations where you actually need both of those things at the same time? C++ tends to get misused a lot, but I feel like the same thing is happening with Rust.
I think that’s an unusual way frame that requirement.
All programs need to handle memory _correctly_. Very few seg fault as part of expected operation.
Is that better?
Personally, I've never met anyone who can ensure memory safety in their C/C++ without onerous restrictions even more severe than those rust imposes, but maybe you're superhuman.
I've done professional c++ for a couple decades. My pipelines usually were conan with some dependencies that were not. The used clang tidy and format, san, cppcheck, coverity and tested release and debug builds with clang and gcc. Coverage was sometimes just clang. But sometimes gcc as well. Tests were usually google test. The team standard IDE was vscode.
Rust has rustfmt which is one standard. It has clippy which in addition to actual defects it forces commin idioms so code looks familiar. It has san and llvm coverage. It has miri. It supports aflop for fuzzing. And it has a single tool for package management and build. Edit: IDE is still vscode and it works great.
The only tooling missing for rust would be formal analysis, which is in work (and which isn't that great a story on c++ either) and gcc. A gcc front end is in work, and there is mrustc to generate c from rust, but yes, gcc front end support would be nice to get all those extra targets without the extra work.
But of course there are use cases where you need memory safety guarantees and bare metal performance. In those cases, sacrificing some memory safety by using Rust is an acceptable tradeoff I think.
All memory-safe languages are built on an unsafe foundation. The Java HotSpot VM is very unsafe C++. That's just how computers work.
You aren't meaningfully sacrificing memory safety by using Rust because the unsafe sections are clearly marked out. That's similar to unsafe C#, Python with ctypes, and many other memory-safe languages.
No, it doesn't. What's interesting isn't so much that random HN posters believe this sort of thing, because hey, who needs to know anything about a topic to post their opinion on a forum right? No, what's fascinating is that this applies to people like Herb Sutter in his "cpp2" language, here's Herb:
> I don’t like monolithic "unsafe" that turns off checking for all rules
Nobody does that. It's possible Herb knows that and is being deceitful but honestly I think it's more likely that without investigating at all Herb has just decided everybody else is an idiot...
fn trust_me_i_dont_need_a_borrow_checker<'a, 'b> (x: &'a mut u32) -> &'b mut u32 {
unsafe { core::mem::transmute(x) }
}
fn main() {
let mut owned = vec![40, 2];
let mut_1 = trust_me_i_dont_need_a_borrow_checker(&mut owned[0]);
let mut_2 = trust_me_i_dont_need_a_borrow_checker(&mut owned[1]);
drop(owned);
let undefined = *mut_1 + *mut_2;
println!("The answer is {undefined}");
}The exact same code, without the transmute (just returning x directly from an unsafe block), fails a type check related to borrowing. Specifically, the compiler can't see why (without a transmute) it should believe that the reference x now has a different lifetime 'b instead of 'a. Which is fair because it doesn't.
The transmute is you claiming it does, and since transmute is an unsafe function that's entirely on you to make sure you're right about that. So, lying has the expected effect.
The borrow checks aren't magic, they can't look into your soul - but they are running inside unsafe code too.
* Edited to make explicit that the alternative is still marked unsafe and yet doesn't work
That, and the borrowing/safety aspects of Rust also shake out into threading/concurrency, where the compiler does a pretty good job of forcing you to adopt better practices in regards to state sharing, locking, concurrency.
Cargo, though... I am coming around to thinking isn't great -- workspace support remains half-assed, especially... and crates.io is amateur-hour for reasons many people have pointed out elsewhere on this post. I have been tempted to switch my personal projects to bazel, with 3rd-party deps checked in/submoduled.
And it offers those things along with a fantastic type system, and great tooling.
This isn't necessarily a problem. C++ and Rust can coexist pretty well in practice, and Rust is a good entry point for learning how to write systems software.
So I guess I wouldn't be surprised that the applications are trending that way, too, as that is where growth is happening right now it seems.
On the other hand, we use Rust on the backend of our web based saas product, and it serves as a great filter for quality developers for us.
Are there any efforts to overhaul or completely rethink this?
Algebraic effects would solve this, but I don’t know any language other than OCaml that is working on that approach.
You can convert colors simply but only one way, and practically that works, functional core imperative shell, etc.
* The async runtime and async functions are decomposed but tightly coupled. That means while you could swap out runtimes, a crate built against one runtime can’t generally be used with another unless explicitly designed to support multiple. I believe C++ has a similar problem but no other major language I’m aware of has this problem - that’s typically because there’s only a single runtime and it’s embedded in the language. Things like timers and I/O are not interoperable because the API you use has to be for the runtime you’re running under. I believe there’s work ongoing to try to remedy this although in practice I think that’s difficult (eg what if you’re using a crate relying on epoll-based APIs but the runtime is io_uring).
* async in traits. I believe that’s coming this year although the extra boxing it forces to make that work makes that not something 0-cost you can adopt in a super hot path.
* async requires pinned types which makes things very complex to manage and is a uniquely Rust concept (in fact I read a conceptually better alternative proposal for how to have solved the pin/unpin problem on HN not too long ago, but that ship has long sailed I fear).
* The borrow checker doesn’t know if your async function is running on a work stealing runtime or not which means there’s a lot more hoop jumping via unsafe if you want optimal performance.
* async functions are lazy and require polling before they do anything. That can be a bit surprising and require weird patterns.
Don’t get me wrong. The effing-mad crate is a fantastic demonstration of the power of algebraic effects to comprehensively solve coloring issues (async, failability, etc). But I think there’s stuff with runtime interop that’s also important. I don’t think anyone is yet seriously tackling improving the borrow checker for thread per core async.
[1]: https://www.unison-lang.org/
[2]: https://www.unison-lang.org/learn/language-reference/abiliti...
In general I’ve noticed that there are a lot of code generation modules that can generate very cryptic error messages if you stay clear of their happy paths.
Any kind of shared resource is a tedious experience, that's basically it's selling point.
It is frustrating becasue if you go look for help, people are always condescending about global shared resources like that but there really isn't a better way.
tl;dr: You can have different (nicer to program) abstractions, but you can't have them in the same language you use to write firmware for a $10 electronic device, or Linux drivers, and we already have languages like Go and Javascript whereas we did not have a safer alternative to C++.
Adding it to your project probably increases the average code quality:)
For typical applications I am learning nim, which has python syntax, c speed and size, and could be memory safe too.
I really feel nim deserves more love, for that you can balance coding-speed(write like python script), size , performance and security, and cal leverage c and c++ libraries without FFI, no other languages can have those at the same time.
Writing performance-sensitive code in C/C++ and calling it via interop used to be the way to go during .NET Framework days but since then has become a performance trap.
Especially for small methods, calling them through interop is a deoptimization because they cannot be inlined, and involve GC frame transition (which you can suppress) as well as an indirect jump and maybe interop stub unless you are statically linking the dependency into your AOT deployment. In case the arguments are not blittable to C - marshalling too. Of course, this is much, much faster than anything Java or Go can offer, but still a cost nonetheless.
It is also complicates the publishing process because you have to build both .NET and C parts and then package them together, considering the matrix of [win, linux, macos] x [x64, arm64], it turns into quite an unpleasant experience.
Instead, the recommended approach is just continuing to write C# code, except with pointer and/or ref based code. This is what CoreLib itself does for the most performance-sensitive bits[0]. Naturally, it intentionally looks ugly like in Rust, but you can easily fix it with a few extension methods[1].
[0]: https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...
[1]: https://github.com/U8String/U8String/blob/main/Sources/Compa...
Good to know you can remain within the framework and get decent performance though with the unsafe pointers/refs. Would be interesting to see a good benchmark using only C# with latest features and Rust, although I cognisant of the fact there is more to it than pure performance (binary size, dependencies, GC etc).
I feel this one, something about spending too much time in JS makes me not want to write clean software..
Lifetimes need something to stand out as an identifier. 'a is weird, but works. Whether it could use a different ASCII sigil is a bikeshed problem.
Types borrowed <> from C++ to look less weird to C++ programmers. But this again is just a surface level issue. Semantically, the wrapper types are incredibly useful. Having all nested types spelled out is convenient when reading code – you know what you're getting and what are the standard properties of it.
I programmed in C and C++ for years, to compiler messages don't make me feel like I should be taking them personally. Sure beats having a runtime error any day.
I wonder if those with that attitude have come from a background where they used dynamically typed languages or non-compiled languages more, or it is something even people with a wider experience find particularly onerous with Rust?
- I loved here. I am still is. But we needed a divorce.
I still like threads, but the I'm old and uncool.
Hey, I resemble that remark.
But I disagree with it. Green threads give you all the advantages of async, but with less of the hairs. In particular no special syntax or change of programming style is required. Yet underneath green threads and async just different styles of event driven I/O, so both run at similar speeds and excel at the same tasks. (Actually green threads should run faster, as storing state on a stack is generally faster than malloc.)
I have no idea why Rust abandoned green threads in favour of async. Actually, that's a partial lie - there have been far too many words wasted on explaining why. The problem is the reasons they give look to be an caused by design decisions they made in their implementation. The primary objection seems to be speed. The current async is indeed faster than their old green thread implementation. But that was caused by their choosing to avoid coloured code in their green threads (maybe they were copying Go?). Other objections were similarly to do with the implementation they threw away, not green threads themselves.
They require more memory over stackless coroutines as it stores the callstack instead of changing a single state. They also allow for recursion, but its undelimited meaning you either 1) overrun the guard page and potentially write to another Green thread's stack by just declaring a large local variable 2) enable some form of stack-probing to address that (?) or 3) Support growable stacks which requires a GC to fixup pointes (isn't available in a systems lang).
> green threads should run faster, as storing state on a stack is generally faster than malloc.
Stackless coroutines explicit don't malloc on each call. You only allocate the intial state machine (stack in GreenThread terms).
> The primary objection seems to be speed
It's compatibility. No way to properly set the stack-size at compile time for various platforms. No way to setup guard pages in a construct that's language-level so should support being used without an OS (i.e. embedded, wasm, kernel). The current async using stackless coroutines 1) knows the size upfront due to being a compiler-generated StateMachine 2) disallows recursion (as that's a recursive StateMachine type, so users must dynamically allocate those however appropriate) which works for all targets.
True, but in exchange you don't have to fight the borrow checker because things are being moved from the stack. And the memory is bounded by the number of connections you are serving. The overheads imposed by each connection (TCP Windows, TLS state, disk I/O buffers) are likely larger than the memory allocated to the stack. In practice on the machines likely to be serving 1000's of connections, it's not going to be a concern. Just do the arithmetic. If you allowed a generous 64KB for the stack, and were serving 16K connections, it's 1GB of RAM. A Raspberry PI could handle that, if it wasn't crushed by the 16K TCP connections.
> They also allow for recursion, but its undelimited meaning you either 1) overrun the guard page and potentially write to another Green thread's stack by just declaring a large local variable 2) enable some form of stack-probing to address that (?) or 3) Support growable stacks which requires a GC to fixup pointes (isn't available in a systems lang).
All true, but also true for the main stack. Linux solved it by using 1MB guard area. On other OS's gcc generates probes if the frame size exceeds the size of the guard area. Lets say the guard area is 16KB. Yes, that means any function having than 16KB of locals needs probes - but no function below that does. Which in practice means they are rarely generated. Where they are generated, the function will likely be running for a long time anyway because it takes a while to fill 16KB with data, so the relative impact is minimal. gcc allows you to turn such probes off for embedded applications - but anybody allocating 16KB on the stack in an embedded deserves what they get.
And again the reality is a machine that's serving 1000's of connections is going to be 64bit, and on a 64bit machine address space is effectively free so 1MB guard gaps, or even 1GB gaps aren't a problem.
> No way to properly set the stack-size at compile time for various platforms.
Yet, somehow Rust manages that for it's main stack. How does it manage that? Actually I know how - it doesn't. It just uses whatever the OS gives it. On Windows that's 1MB. 1000 1MB stacks is 1GB. That's 1GB of address range, not memory. Again, not a big deal on a modern server. On embedded systems memory is more constrained, of course. But on embedded systems the programmer expects to be responsible for the stack size and position. So it's unlikely to be a significant problem in the real world. But if does become a problem because your program is serving 10 of 100's of concurrent connections, I don't think many programmers would consider fine tuning the stack size to be a significant burden.
> No way to setup guard pages in a construct that's language-level so should support being used without an OS (i.e. embedded, wasm, kernel).
There is no way to set up the main stack without the kernel's help, and yet that isn't a problem? That aside are you really saying replacing a malloc() with mmap() with the right flags is beyond the ken of the Rust run time library authors? Because that is all it takes. I don't believe it.
> The current async using stackless coroutines 1) knows the size upfront due to being a compiler-generated StateMachine 2) disallows recursion (as that's a recursive StateMachine type, so users must dynamically allocate those however appropriate) which works for all targets.
All true. You can achieve a lot by moving the burden to the programmer. I say the squawks you see about async show that burden is considerable. Which would be fine I guess, if there was a large win in speed, or run time safety. But there isn't. The win is mainly saving on some address space for guard pages, for applications that typically run on 64bit machines where that address space address space is effectively an unlimited resource.
The funny thing is, as an embedded programmer myself who has fought for memory I can see the attraction of async being more frugal than green threads. A compiler that can do the static analysis to calculate the stack size a number of nested calls would use, set the required memory aside and then general code that so all the functions use it instead of the stack sounds like it could be really useful. It certainly sounds like an impressive technical achievement. But it's also true I've never had it before, and I've survived. And I struggle to see it being worth the additional effort it imposes outside of that environment.
Wow, as the author of that feature, I'm surprised to see someone was so passionate about it. I've found that many times I've been having to tell people why they should care about it.
> I don’t know why. Maybe the pressure to maintain stable APIs, along with Rust’s granular type system, makes it difficult for library owners to iterate. It’s hard to accept a minor change if it would result in a major version bump.
There is a tension between people wanting features and people not wanting to deal with version bumps. I've seen this a lot in maintaining clap, especially when it went from unmaintained for years to having active releases.
As for cargo, the compatibility guarantees are tough. Take the lints table. We can't throw in a first try, knowing we can fix in in a cargo 2.0. We are tied into the rust project itself which means we have the same compatibility guarantees. This is one reason we generally encourage trying ideas out in third-party plugins before we integrate them in directly since they can break compatibility.
> You can’t even publish a crate that has local dev dependencies
You can; cargo automatically strips them. However, if you tell cargo that there is a version of it in the registry (by setting the version), then it must be published. This is why when I redesigned `cargo add` for being merged into cargo, I made it so `cargo add --path ../foo --dev` will not add the `version` field. We do need to find ways to clarify that the purpose of the version field is for looking it up in the registry.
Allowing the dev dependencies to be stripped also helps with issues of circular dev-dependencies.
> However, many developers break large projects down into smaller modules naturally, and you can’t publish a parent crate that has sub-crates that only exist within itself.
We do have an RFC for this: https://github.com/rust-lang/rfcs/pull/3452
The most complex part is the Index, figuring out how to represent it in the metadata tables we maintain so we avoid having to download every `.crate` file.
I also worry there might be tech debt around assumptions of there being a single version of a package when nested packages will easily break that.
> You can see the problem manifest in the sheer number of utility crates designed to simplify publishing workspaces. Each works with a subset of configurations, and the “one true way” of setting workspaces up still eludes me. When I publish Wick, it’s frequently an hour+ of effort combining manual, repetitive tasks with tools that only partially work.
I'm a bit confused on this point. While there are things to improve around publishing workspaces, I'm not sure how this relates to setting workspaces up or what problems they've had with that. I'd also be curious what problems they had with releasing packages. I don't think I've seen issues from them in cargo-release's Issues.
One thing I don't appreciate is nonorthogonality of certain uses and control structures that cannot be refactored into a function despite structural equivalence without introducing a multiple borrow conflict.
Need performance, security, and reliability? Build that part in Rust, and have the rest be executed and orchestrated in python. It also forces great design patterns in the form of encapsulation and a strong API.
I think the idea of being monolingual in codebases is a silly limitation and lots of development teams would be a lot more productive if they embraced the idea of polylingual codebases.
Years ago I tried compiling a rust sample program. The binary size was one thing that put me off.
Today, I see the issue of large binary size has been addressed. The binary I get is sufficiently small.
If this issue of ~500M standard library is fixed, e.g., if some of it is made optional not mandatory, I will give rust another try.
3 packages will be downloaded:
3 packages will be installed:
libexecinfo-1.1_3
libexecinfo-devel-1.1_3
rust-std-1.73.0_1
Size to download: 136MB
Size required on disk: 509MB
Space available on disk: never enough!
The simple question is how much space is required for an installation of the Rust compiler. On VoidLinux, it's significantly more space than for a GCC installation. 509MB is just too much for my tastes.
Had this experience in my first Rust program. It turned out Rust had silently changed the types of some variables based on some code hundreds of lines later, which caused an overflow at runtime. Spooky action at a distance!
I'm curious if this is other's experience as well. I've only dabbled in Rust but it isn't clear to me how it would ensure that the code is correct.
No. This is an exceedingly rational anger.
But maybe I’m bikeshedding by saying that.
This made me not want to try Rust. I was coding vanilla JavaScript for a very long time before trying TypeScript but when I did I was amazed at how much easier refactoring became. That alone made me very excited about type systems and I instantly started looking at Rust, thinking it would be even better. Apparently not. Maybe the sweet spot (for me) is TypeScript.
The carbon docs never recommend rust, they just urge you to use something already available.
This is a pretty negative and unfortunate take on rust that I cannot recognize after having spent the past 2 years professionally writing backend rust. I did write c++ before that so it's likely that it takes some experience with the pain of an unhelpful (and in some cases downright hostile) to fully understand and appreciate the so called "yelling" rust does.
From my perspective every lifetime complaint the compiler has is a deeply appreciated hint that not only saves you countless hours of debugging down the line but also makes me feel confident/safe in the code which you never could achieve in a c++ codebase without making defensive copies everywhere.
Being able to lend/borrow data without fear truely is rust's greatest strength and something people coming from GC languages have a hard time appreciating.
One kind of pain has a bigger blast radius than the other. I prefer compile time pain to runtime pain.
I would find that argument passable if some of the biggest products from the largest companies functioned better. Performant software seems like a distant memory.
In startups that chasm is generally filled with VC money paid to cloud vendors, but if you're bootstrapping or not looking to be a unicorn, you probably will want to drive down infrastructure costs much earlier.
One need only open Microsoft Teams to see the “billing time” costs. If the billing time cost is not paid by the developers or their company, then it stop being a cost to them and it becomes an unpriced externality. Who cares, who even knows, that this app we’re writing runs slowly on regular people’s old hardware.
Or if you write mobile app and became slow and laggy, then you lose users.
Was that the result of a process in which the compiler “abused” me? I don’t think so.
Meh, the fact that Rust's compiler is overly strict is just a truth (All compiling programs are valid, but not all valid programs are compiling), and I find it lowkey annoying that every time someone has a problem with it, rustceans jump to the defense to the point of almost denying the OPs experience. I mean on a case by case basis it might simply be a difference of experiences and both are valid, but on a larger scale it does look like a bit of a pattern, at least to me.
Ofcourse, there restrictions aren't just a whim of a Rust core team (usually ;p ), and do come from practical limitations but regardless, it's fair to be frustrated at them.
The author also wrote >[Rust's] dev tooling leaves much to be desired
I can't even begin to comprehend such a statement. Rust's tooling ecosystem is the number one reason why I want to convince my colleagues to give it a shot. Its literal heaven coming from C++ with cmake...
I need to use ChatGPT as a programming sidekick/pair programmer/mentor. Yes, I am not afraid to admit - I need an AI assistant and won't program without it - AI is far too valuable.
If ChatGPT struggles with a programming language that is a big problem.
I have had a great experience with ChatGPT with TypeScript, SQL, Python, golang, C++.
I tried ChatGPT with Rust and it essentially failed - it does not understand the language.
AI has become an essential tool for me in programming if the language doesn't do AI or doesn't do it well then that's a huge mark against it.
Not knowing the hard parts is not a workable outcome.
I understand Rust pretty well. I can totally see ChatGPT not being able to help with complex lifetime issues. I’ve never asked it such things.
Pasting in an error message, made chatgpt spit out another version of the code with some other error.
Run it. If there is an error, either debug it or ask ChatGPT to take a different approach.
There's something wrong if you can't get it to do this.
Of course, it's still an issue if ChatGPT can't generate trivial rust code, but that seems unrelated to the quality of the language itself.
Yes, there's something wrong here. But it isn't "not using chatgpt"...
Plus, I'm way more trusting of a Stack Overflow post or even a blog post than what AI generates. I mean, AI hallucinates all the time or generates things that are subtly wrong, so unless you can write the code yourself how can you trust the generated stuff?
You're not using the tools available to make you a better and more productive programmer?
If you're not using AI as part of your daily workflow you are burning time and money and not maximising your potential.
If I was your team lead I'd give you a warning, same as I would if you didn't use an IDE or source control.
Being ignorant of programming productivity tools isn't something to be proud of.
You should have been up to speed by now on using ChatGPT as part of your daily workflow - it's been out since the start of the year.
I'm not averse to new tools at all, but I've yet to see where I would want copilot or chatgpt. The problems I work on daily, like most professional devs, are very specific to how I adapt a large proprietary codebase to do new things to fit specific business requirements, or working with designers and project managers to figure out how we should solve these intricate problems together. AI can't help with that. It'd take a 20 page prompt for it to roughly understand the business even.
It can help with toy problems like how to write a well known algorithm, but given that these are well known algorithms and it's basically just copying from open source repo's (with scary legal ambiguity), it's of little use to what I actually do day to day.
I'm amused that as a lead you'd "give me a warning ". I've literally never worked with a lead or manager that cares what tools people use, only that the work is good. Actually, if anything a lot of workplaces are asking people to seek approval before using AI, because of the many thorny issues (copyright being one of many)
(BTW, I do use chat gpt, just not for coding. It's useful for creative tasks or summaries. I still google what it says because it does make shit up)
I really think you mix up
a) common rules as they are needed to work together and (like style or source control) b) local tooling / someone is organizing his own workspace
including an dictator-like "MY way of working is the best!" attitude after disovering whatever works out for YOU. If I was your team lead, I guess I would give you a warning. :-D
Additionally, I guess your programming problems are very generic.
In The Grapes of Wrath, Steinbeck writes about the travails of a family's trip to California in search of work during the Great Depression in the 1930s. Tom Joad, the father, fixes the compression in his blown engine by wrapping a copper wire around the cylinder, then running the motor until it melts and recreates the seal. It's such an ingenious hack, and reflects such deep knowledge of how an engine works, that it's stuck with me for decades.
Today, I bet fewer than 5% of vehicle owners even know how to check their tire pressure. But it's not like fewer people are driving, or cars are more likely to be found by the side of the road with a blown engine—it's just far less necessary to know these things. I think programming will go this way, too.
[0] https://en.wikipedia.org/wiki/Adiabatic_flame_temperature
the problem with that story is that if the "seal" is made from a simple material that melts by the engine heat, then it will not seal for long. (or at all.) but likely it's not what Steinbeck wrote.
I seem to remember my dad shimming a cylinder that had lost compression with a copper ring; I was a kid so I don't remember any details except it wasn't intended to melt, and whatever it was intended to do worked and we saw the car still being driven around town fifteen or twenty years after he sold it.
copper becomes ductile between 300-600 C ( https://www.engineeringtoolbox.com/docs/documents/1353/tempe... )
piston ring seals are usually made of cast iron or steel. copper would probably work too for a while, it's good at conducting heat, there's ample cooling in engines, so it wouldn't melt, just wear out very quickly ... and then the engine performance degrades as the sealing gets worse and worse (and it starts to eat oil, soot gets everywhere, exhaust becomes visible), mpg goes down, but ... the car would probably run. (loss of 1 out of 4 pistons is not a catastrophic failure)