Hacker News Reader

Comment by matthewbauer | Hacker News Reader

Rewriting stable software in Rust is a very bad idea. Especially in open source, where we don't have enough maintainers, it ends up hurting the ecosystem. A certain prominent GNOME maintainer has been trying this recently and breaking things along the way:

https://gitlab.gnome.org/GNOME/librsvg/issues/456

Apparently, GObject Introspection doesn't even work in Rust, yet it is expected to be a perfectly valid replacement:

https://github.com/gtk-rs/gir-files/issues/35

Please don't be like these people! Keep the stable software we have, and maybe write new software in Rust.

View on HN

Your argument is not compelling: you've described the drawback without considering the benefits. What if that prominent maintainer hadn't made the mistake (a mistake attributable to poorly captured design or requirements, but not the implementation). Is there something intrinsic to Rust that would lead them to this error? (no, I think not).

> Please don't be like these people! Keep the stable software we have, and maybe write new software in Rust.

I say: spend your time how you see fit. I love Open Source software, I love GNU/linux and GNOME. But IMO it's always been about scratching an itch first and commitment to a cause second.

It's certainly up to them if they want to rewrite everything in Rust. I'm just saying the net benefit just does not exist for stable software like librsvg or bzip2.

> Is there something intrinsic to Rust that would lead them to this error? (no, I think not).

There is nothing intrinsic to Rust that creates this kind of problem, besides it being a different language than the project was originally written in. I'm sure Go, Swift, Haskell, Java, or any other language (with the possible exception of C++) would have similar issues. No other language community is quite as arrogant as the Rust community though.

> I'm just saying the net benefit just does not exist for stable software like librsvg or bzip2.

Are you sure?

Both of those libraries are, in fact, poster children to be rewritten in Rust as they have to eat untrusted input.

If I offered $20 on "I can find a use-after-free or undefined behavior bug somewhere in those libraries" I'm pretty sure nobody would take that bet.

To be faire, that is not everywhere.

The design team always has nice conversations with me, in spite of my schizophrenic view of C++ vs Rust (like both languages, see some negative issues in both), and there are places with joint community events between C++ and Rust.

> Please don't be like these people!

As the maintainers of the software shouldn't they have the prerogative to maintain the software in the best way they see fit?

Bugs happen while refactoring, regardless of how the refactoring is done.

They do, but that doesn't mean the decision is wise. The point is you are creating more bugs than you could ever possible fix in this kind of refactor.

And those bugs get fixed, and the software ends up better.

I've contributed to two successful "rewrite it in Rust" projects now: Stylo and WebRender. Both of them ended up fixing long-standing bugs in the previous implementation that were difficult to address in the old codebase, but a new clean approach offered a nice opportunity to fix them.

I also err on the side of never re-writing things, and pros and cons must always be weighed on a case by case basis. It's not possible to say in general that re-writing software in a language not prone to many important classes of bugs would "create more bugs than you could ever possibly fix." We won't know the decision is wise or unwise until it's attempted and studied maybe even a few times over.

If you're rewriting into a language like C++, this may very well be true. However rewriting in Rust in my experience yields far fewer bugs, and the ones that do surface are usually simple to fix. If a project truly is "stable" - there's very little effort involved in maintaining it, then yeah, it doesn't make sense to rewrite it. But if it's plagued by bugs and is painful to maintain, you'd be much better off rewriting it in Rust, assuming you're familiar with the existing project's limitations.

Rewriting stable, network-facing, C or C++ software in Rust is a very good idea, because that way that software will require less ongoing maintenance to avoid security problems in the future.

We don’t know yet how much Rust software will cost in ongoing maintenance to avoid security problems.

Unsafe keyword disables many safety features. It already caused security issues: https://medium.com/@shnatsel/how-rusts-standard-library-was-... That particular one was fixed long ago, but unsafe is used a lot in libraries, both in standard and third-party crates. The reasons include native interop e.g. OS kernel calls, language limitations e.g. in collections and other data structures, also sometimes they’re performance optimizations.

Yes, we do. Empirically, there have been far fewer memory safety problems in the Rust components of, for example, Firefox than there have been in the C++ components.

That particular memory safety issue was not a security issue, because it did not cause any problems in actual software. Rust is very conservative about issuing CVEs for any issue which could conceivably be a security problem. (The equivalent would be if C++ the language issued a CVE because the language lets you clear an array while you're iterating over it. The CVEs for exploitable problems that this kind of thing enables are for the vulnerable software, letting C++ off the hook.)

In my opinion, Rust is too conservative and shouldn't even call those things security issues until they affect real products, because it leads to misunderstandings like this.

It’s safer than C++, sure, but half of other languages are safer. Java doesn’t have unsafe code at all, not even in standard library, and it’s very hard to interop with. Same with JS & TS.

When people move from C++ to another language for more safety, Rust is not their only option.

Couple decades ago everything was written in C or C++. Desktop apps, mobile apps, web servers (cgi-bin, then COM objects for asp.classic).

People wanted higher level, easier to use, faster to compile, and safer languages. Java was the first popular one, then C#, then the rest of them followed. C++ is continuing to lose the market it once had. 15 years ago, people generally stopped using C++ for web servers. Some still do for unusual requirements, but most people don’t. 10 years ago, most people stopped programmed mobile apps in C++ (I did before that time, for WinCE and PalmOS), now it’s mostly managed languages there. Recently the same happened for desktop apps, no one likes Electron but it does the job, and people do use the software. Embedded and videogames have already started the transition, IMO.

Moving from C++ to better languages is a long trend, the industry is doing it for decades now. I don’t think Rust is a universally good option for that. Has issues with usability, labor market, libraries, platform support, and community. Good one for niche stuff, like some components of a web browser, or a bare metal hypervisor, but that’s it.

> Java doesn’t have unsafe code at all, not even in standard library, and it’s very hard to interop with.

Isn’t all JNI code in Java considered unsafe?

Then there is http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot...

JNI is what I meant by "hard to interop with".

That second thing is fine I think, the docs say "Although the class and all methods are public, use of this class is limited because only trusted code can obtain instances of it."

Similar policies are in .NET and they work well IMO, you can compile a .NET DLL with /unsafe but most people leave default options, then both compiler and runtime verify that it's true. Other way works too, set up the VM to partial trust and you won't be able to access native memory at all, except memory in managed arrays and structures.

But this isn’t different in Rust. unsafe is generally only reserved for low level features. In the trust-dns libraries, there’s not a single line of unsafe code, whereas in the Postgres extension library I’ve been working with some others on there’s a ton of unsafe for ffi with the C. Even with that, the intention is that for most users, they’ll never have to write unsafe.

Just because of the presence of unsafe (which still guarantees more in Rust than C), doesn’t mean that the entire codebase is unsafe.

This is a nitpick that is used to malign Rust, but really it’s a freedom to implement as low or high level functionality as you want, with out cumbersome interfaces like JNI or even JVM features in Java.

.NET or Java sandbox is extremely hard to escape unless you have a door open. It's borderline impossible. Their byte code is simple, the runtime has a chance to check which classes are created or what pointers are accessed. It then has complete supervision over the program as it's running.

Escaping Rust sandbox is a short keyword away. That keyword is used all over codebase, in both standard and third party libraries, to implement these lower-level features, or overcome language limitations. You can't disable it because you'd loose vectors and strings.

> really it’s a freedom to implement as low or high level functionality as you want

I'm a C++ and C# developer. I often use both in the same software, and I think my difference in levels is wider. On the lower level I have SIMD intrinsics, simple but very fast unsafe data structures, manual control over RAM layout. On the higher level I have LINQ, async-await has been working for a decade now, reflection.

> with out cumbersome interfaces like JNI

Here it's not too bad. C-style interop i.e. [DllImport] just works, even on ARM Linux where it loads .so libraries. On Windows, COM interop allows to export C++ objects from C DLLs, or expose .NET objects to C++, or move ownership either way. Some boilerplate is required on Linux to expose OO APIs without IUnknown.

> Escaping Rust sandbox is a short keyword away. That keyword is used all over codebase, in both standard and third party libraries, to implement these lower-level features, or overcome language limitations. You can't disable it because you'd loose vectors and strings.

There is no meaningful difference between the JNI and the HotSpot VM and the Rust unsafe keyword. The HotSpot VM is hundreds of thousands of lines of unsafe C++ code. It doesn't get a pass because it's the VM.

Rust uses unsafe to implement things like hash tables and lists, primitives that would otherwise be implemented in unsafe code in the compiler, because it was easier to write code than to write code that generates code. I actually hand-coded early versions of Vec in LLVM IR in the compiler. It was a miserable experience! Moving the implementation to unsafe code instead of raw LLVM IR made it easier to maintain, which made it safer.

> There is no meaningful difference between the JNI and the HotSpot VM and the Rust unsafe keyword

JNI: used rarely. Can opt out, the runtime will enforce the policy, disabling all unsafe code. Most third-party libraries don't use nor require it.

JVM: Identical prebuilt binaries are installed by millions of users. Developer is a for-profit company who's existence depends on the security of their product. Only small count of people can technically change the unsafe code. Can't opt out this will disable Java.

Rust unsafe: the unsafe code is used in half of the libraries, first and third party, i.e. authored by very large group of people. There's no single binary, potential bugs are spread across entire ecosystem. Test coverage varies greatly, Vec from stdlib is used by everyone and very likely fine, but a long tail of unsafe code not used in practice much, or at all. Can't opt out this will disable Rust.

The difference is quantitative, in my estimation of risks. I don't exclude someone has deliberately placed a security critical bug in a JVM using some real-life bug (blackmail, bribe, etc.) but other people are IMO much more likely to notice, than if similar security bug is in some obscure Rust crate. It's same with human errors.

> Rust unsafe: the unsafe code is used in half of the libraries, first and third party, i.e. authored by very large group of people. There's no single binary, potential bugs are spread across entire ecosystem. Test coverage varies greatly, Vec from stdlib is used by everyone and very likely fine, but a long tail of unsafe code not used in practice much, or at all. Can't opt out this will disable Rust.

I'm glad you agree that unsafe code in the standard Rust library is fine.

Concern over unsafe code in dependencies is a legitimate concern, but it's one that we have tooling for, such as cargo-geiger. With that tooling, you can opt out of unsafe code in dependencies in Rust just as you can in Java. (Note that unsafe code isn't the biggest potential problem with dependencies, though. A malicious dependency is a serious issue, regardless of whether it does pointer arithmetic or not.)

Besides, this is basically splitting hairs. Empirically, we know that memory safety problems are incredibly rare in Rust programs compared to those in C++ programs. Maybe Rust programs have slightly more memory safety problems than Java programs do, or vice versa. The difference isn't a meaningful one.

> I'm glad you agree that unsafe code in the standard Rust library is fine.

I don't. I have no doubts in Vec class because it's used a lot, not because it's in the library.

I'm pretty sure the standard library also has that long tail of barely tested unsafe code.

I've recently looked at sources of stdsimd crate, it has hundreds of unsafe functions doing transmute. I code SIMD intrinsics a lot in C++ and I know they're probably OK. Apparently, wasn't even for pointer math, the only goal of unsafe was to workaround a language limitation (they're pure functions without side effects)

> A malicious dependency is a serious issue

I never found any, but I found bugs in my dependencies many times. Not fun either.

> memory safety problems are incredibly rare in Rust programs compared to those in C++ programs

I agree, quite expectable. BC works two ways, it checks ownership at compile time, also raises entry barrier. Both help with memory safety. The observations don't tell which effect contributes more.

There are JVMs written in Java, and in fact that is what Project Metropolis is all about, take code from GraalVM and increasingly replace that C++ in OpenJDK with more Java code instead, including defining a so called System Java subset to be AOT compiled by SubstrateVM.

You still have to deal with both the #1 and #2 worst ideas in computer science — nullability and inheritance.

You don't have to.

C# has value types incl. user ones, they are not nullable.

If you don't like inheritance, don't use it. I sometimes write pages of code without any non-static class. When doing OO, if you want polymorphism but don't want inheritance, use interfaces. They are not base classes, you can implement many, or implement on a value type.

> C++ is continuing to lose the market it once had.

This is an amusing remark. The number of C++ programmers is growing faster than ever, C++ discussion boards are busier than ever, and attendance at C++ conferences is higher than ever, and the growth rate is increasing. The number of C++ programmers is growing at a far, far greater rate than of Rust programmers.

It's the difference between fads and real trends. C++ is where the hard work gets done.

The language had a renaissance in 2011, and is no longer what it was. Moving to a better language is easier than ever, because C++ is that language. C++14 is better than C++11, C++17 is better than C++14, and C++20 will be way better than C++17.

Talk about safety is misplaced. Modern C++ is as safe as any language you can name. Of course there are (literally!) many billions of lines of old code, most of it nothing to write home about. But it's overwhelmingly easier and safer to modernize it than to rewrite it. Save rewriting for C, where it makes the most difference; then, rewrite it incrementally in C++, keeping it fully functional at all times.

The safest code is code you don't have to write at all, because it's in a fast, powerful, well-tested library you can just call. C++, overwhelmingly more than other languages, is focused on enabling powerful libraries you won't be tempted to bypass, for speed or because it might be a PITA to use. As a direct consequence of that focus, we have the libraries.

Talk about Java or C# as "better" languages makes me laugh. The number of bugs is proportional to the lines of code. Such infamously prolix languages provide fertile soil for bugs, even neglecting their poorer library support.

Rust is an interesting modern language that is relatively good for writing libraries in. It is a sound choice anywhere that the alternative would be C, Go, or Java, provided tool maturity and coder availability are not serious concerns. It is a somewhat risky choice, because it is still far from clear whether it will still be growing in five years, but so far so good.

> Modern C++ is as safe as any language you can name

It absolutely isn't. C++ will always have the looming spectre of undefined behaviour.

I've seen repeated instances of undefined behaviour in respected textbooks: initializing a struct by zeroing it out with memset. It'll probably continue to work fine on MSVC/x64, but presumably the (very knowledgeable!) author didn't realise C++ doesn't guarantee that null will be bitwise zero. Or perhaps they didn't care, and were fine with introducing undefined behaviour into their example code (the kind of sloppiness that demonstrates the value of safe languages).

I suppose you could argue that isn't modern C++, but that would be weak sauce given that we're discussing the safety of the language.

> When people move from C++ to another language for more safety, Rust is not their only option.

That's arguably missing the point. For a very long time, managed, GC-based languages like Java or Go were the only option for workable memory safety! (Yes, I'm ignoring a lot of stuff because it's practically irrelevant. No need to tell me about ATS and the like, thank you so much.) Cyclone and now Rust have changed that, which makes for a real paradigm shift in the industry, at least potentially. Rust even contributes meaningfully to thread safety by getting rid of simple data races, which are a huge can of worms not just in C/C++ but managed languages as well!

> which are a huge can of worms not just in C/C++ but managed languages as well

Multithreading with mutable shared state is harder in general. That's language agnostic.

But it's easier to do correctly the way managed languages do. They don't usually create threads, they use thread pool and post messages to each other. This is language-agnostic too, I've tried doing same in C++ and it works, e.g. https://github.com/Const-me/vis_avs_dx/blob/master/avs_dx/Dx... but with a GC and async-await it's much easier.

> which makes for a real paradigm shift in the industry, at least potentially

I think for most people, getting stuff done is more important than shifting paradigms.

Rust is less productive than managed languages (GC helps, BC interferes), compiles slow, slightly less safe.

For network people, golang is quite popular already, .NET core wants there, too. Game developers use too much C++ libraries, don't need security (console games are sandboxed or run under HW hypervisor), their most painful C++ issue is build time, and Rust is even worse. Low-level embedded, driver, linux OS kernel developers need C, too much integration and too many weird target platforms. HPC and ML people need manual SIMD and corresponding libraries. CUDA people need nVidia support.

Open a job board, type C++ and press search. Do you see many positions except listed above, where people are still writing C++?

"Mutable shared state" is precisely what the Rust compiler checks for. Mutable state (in sequential code), no problem. Shared immutable data, OK. But as soon as you inadvertently mingle both the compiler will notice, and demand that you deal with the issue (by explicitly serializing access, and/or by using lightweight runtime checks that ensure you're not "sharing" anything that shouldn't be - this is what happens when you use Rust's 'interior mutability' facilities).

And because these places are explicitly marked in the code, it's easier to audit them for bugs, same as with 'unsafe'. "Posting messages" is just another way of sharing non-mutable data, Rust has facilities to help you do that as well.

By the way, this is why the "borrowck interferes" quip is only true if you haven't internalized the things BC checks for. Once you have done so, you understand how it helps. The slow compiler is an issue unfortunately, but that only affects code generation - the static checks Rust does are quick and easy. Besides, "if it compiles it works", right?

> is precisely what the Rust compiler checks for.

Mutable shared state is required in many real-life cases. CPUs suffer heavy penalty for random RAM reads, random writes are free (especially when using MOVNT* instructions to bypass caches). When working on computational code you want to optimize for read access. This often means write patterns are random, i.e. you can’t slice arrays per compute thread.

> because these places are explicitly marked in the code, it's easier to audit them for bugs

C# has very good OOP support. All class members are private by default. Wrap that mutable shared state into a class with thread safe public API, will be equally easy to audit.

> But it's easier to do correctly the way managed languages do. They don't usually create threads, they use thread pool and post messages to each other.

C# certainly never worked like that. `System.Threading.Thread` was the backbone of its concurrency model, together with everyone’s favorite auto/manual reset events, mutexes, and semaphores. While with .NET Core there are some libraries providing channels seeing adoption, the majority of the decline of threads in both legacy and Core .NET is from the move to async everything, which really obviates most of the need the casual developer has for dealing with concurrency primitives (at the cost of being absolutely dumbstruck when the abstraction leaks, obviously).

Redmond did an incredible job updating basically every single API in the BCL for async (contrast to the pathetically slow going in the world of nodejs).

> C# certainly never worked like that.

https://docs.microsoft.com/en-us/dotnet/api/system.threading... scroll to the bottom and you'll see it's from .NET 1.1. That's 16 years ago. In modern world, Task.Run does approximately same.

> together with everyone’s favorite auto/manual reset events, mutexes, and semaphores.

When I'm not sending messages, I prefer lock() keyword. A syntactic sugar over Monitor class. Unlike events and semaphores, it's not a wrapper around an OS sync.primitive, but implemented on the VM level. Pulse and TryEnter APIs are nice.

> incredible job updating basically every single API in the BCL for async

The important job was done much earlier then that. Before .NET 1.1 MS decided the fundamental IO class, Stream, needs async API, exposed BeginRead/EndRead, and the rest of them. That's how updated to async-await that fast, they are thin wrappers over the underlying async API which was in standard library for years already, tested and debugged. Few people used them before because callback hell was bad.

> The important job was done much earlier then that. Before .NET 1.1 MS decided the fundamental IO class, Stream, needs async API, exposed BeginRead/EndRead, and the rest of them.

I think the reasoning for this goes back even further. Windows NT already offered a set of comprehensive async APIs in term of IO completion ports and overlapped operations. The .NET APIs are merely wrappers of those. They are too good to omit them.

> he majority of the decline of threads in both legacy and Core .NET is from the move to async everything, which really obviates most of the need the casual developer has for dealing with concurrency primitives

I would argue that async/await in languages like C# (and Kotlin - and even Rust!) makes things even harder for the casual developer. Since async functions are scheduled between a variety of threads developers must now keep track of 2 different concurrency units: Tasks and Threads. Depending on where continuations are called (which is quite complex in C#s world of SynchronizationContexts and TaskSchedulers) the rules are slightly different. And misusing synchronization can have the effect of anything between race conditions (as in normal threaded systems) and starved systems (because the programmer blocks the whole scheduler - which is unfortunately a quite common issue in async/await code).

In my opinion the simple languages are the ones which don't offer 2 worlds. Singlethreaded models (like in Javascript and Dart) are the simplest, since they offer only one model since the cooperative runtime prevents lots of races right from the start. The slightly less simple thing is only offering a good threaded model, which is what Go does with a lot of success. Here we at least only have to learn the old fashioned synchronization primitives, plus some new ones (Channels). Multithreaded languages with support for async/await are absolutely awesome tools! But I think mastering them requires more understanding compared to languages without those features.

I agree that there are many other fine languages that people should be considering.

> Java doesn’t have unsafe code at all, not even in standard library, and it’s very hard to interop with.

Look at Android system libraries. They use JNI all over the place.

You mean like using unsafe all over the place in many third party Rust libraries?

Also Android is a poster child that Google hiring practices don't lead to the best quality of code implementation.

> You mean like using unsafe all over the place in many third party Rust libraries?

Yes?

I don't think Rust is particularly more, or less, memory safe in practice than Java. They're both memory-safe languages.

Ah sorry, I misunderstood your remark then.

It's a very good idea because you don't have to hire or retrain the people to perform the work and maintain it.

Instead you can just talk about how it'a a good idea.

Microsoft security unit also thinks that way.

https://www.youtube.com/watch?v=PjbGojjnBZQ

https://github.com/microsoft/MSRC-Security-Research/tree/mas...

"Future development should be a mix of C#, Rust and constrained C++."

Which is also a reason why they aren't that bothered into keeping plain old C around on VC++, beyond ISO C++ requirements.

Microsoft say a lot of things. I'll consider listening when they will (re)write their stuff in Rust.

TBH what MS does development-wise is irrelevant for me, since I gave up on their platforms a long time ago. But if they put their money where their mouth is, it might give Rust the push it needs.

Wouldn't count on it though, MS can't even figure out what own development story to support for more than a couple of years.

Then start listening.

Azυre IoT Edge is written in C# and Rust, Visual Studio Code uses Rust for its search, the Web framework Actix is written by Microsoft employees and used internally at Azure, they are helping port Firefox to HoloLens and that included hiring Rust devs.

First of all, thanks for letting me know which Rust projects MS are directly or indirectly involved in.

It's more than I expected, but only because my expectations were very low.

Do you consider that list in any way impressive or representative for the MS organization? I mean writing the search function of their hobby project text editor in language X is not exactly a ringing endorsement. Neither is anything which has IoT in its name.

Microsoft security advisor for new projects is C#, Rust and C++ with Core Guidelines profile.

So yes, I find relevant that an IT gigant actually cares to spend money on Rust, regardless how much.

Rome wasn't built in a day.

Thanks for needlessly personalizing the argument, but I've worked on two "rewrite-it-in-Rust" projects that were successes, shipping right now: Stylo and WebRender.

Third parties pressuring the people doing the work to do more work to satisfy the criteria of said third parties => not ok.

1st parties deciding to rewrite their projects, like in your two examples => sure, it's their projects.

There is a special case when new projects are started in C or very security-critical projects are started in C++.

Here public interest should take precedence - it's reasonable to criticize the creators of such a project.

Rewriting stable software is a bad idea, but not because we have too few maintainers. Indeed, the way of dealing with the "too few maintainers" problem is to bite the bullet and start aggressively paying down technical debt so that the ecosystem can (1) become more sustainable going forward, given the same amount of maintainers, and (2) attract more people to the job of being a maintainer, by easing some of the hardest parts of that job. Rewriting stuff in Rust is one way - though a highly risky, perhaps even extreme way - of paying down technical debt.

Rewriting is not like paying down technical debt, it's declaring bankruptcy and trying to start anew from the ashes.

Or depending how it's done, like abandoning the debt and starting a new life on a tropical island under an alias.

Far from seeing this as a downside, I see it as an upside.

Gtk support on OS X has been absymal, forever. If this helps that, I'm all for it, thanks.

Reply on news.ycombinator.com