Comment by tschellenbach | Hacker News Reader

Comment by tschellenbach | Hacker News Reader

2021 will be about them switching back. All jokes aside, it's really common for engineers to randomly switch between solutions incurring huge costs for the companies they work for. Often as it is in this case with no gain for the company, other than some resume building for the involved team members. We use Go here at Stream and power activity feeds and chat for a billion end users. Works perfectly.

Cockroach also wrote about GC optimization: https://www.cockroachlabs.com/blog/how-to-optimize-garbage-c...

For caching specifically there is this library as well: https://dgraph.io/blog/post/introducing-ristretto-high-perf-...

Nothing wrong with Rust, exciting project, but personally think Go is better for building performant micro services like this. Rust seems more suited for building browsers, drivers etc. Rust's performance is slightly better, but it's not as refined around go routines and developer productivity.

Edit: On the other hand, Discord could have made the very valid decision that they have an easier time recruiting for Rust than for Go. Which is in fact a good reason to switch.

Not a chance. We are going hard on rust in 2021 after some very successful projects in 2019 and 2020. our engineers have ramped up on the language - and we have good support internally (both in terms of tools, but also knowledge) to see its success.

Once you've passed the learning curve - imo, rust is far easier and more productive to write than go - especially if you know how to leverage the type system to build idiomatic code and apis that are very hard to use incorrectly. Every piece of rust code we have shipped to production so far has gone perfectly thanks to the really powerful compile time checks and guarantees of the language. I can't say the same for our experiences with go.

Our reasons go well beyond "oh the gc in go has given us problems" but more like "go as a language has willingly ignored and has rejected advances in programming languages". You can pry those algebraic data types, enums, borrow checker, and compile time memory management/safety, etc... from my cold dead hands.

And for network programming - I've been incredibly delighted with the power and flexibility of rusts futures combined with tokio. The fact that you can just drop a future to cancel it, as opposed to having to thread around a ctx which may or may not work has been amazing.

So there is also no point keep using Elixir then.

We will continue to use beam/otp where it makes sense. In fact some rust runs in BEAM now thanks to Rustler. We wrote a blog about this!

https://blog.discord.com/using-rust-to-scale-elixir-for-11-m...

You said that Go doesn't have: algebraic data types, enums, borrow checker, and compile time memory management/safety which Erlang / Elixir is even worse in some of those because it's a dynamic typed language so it's a bit confusing.

I think you're missing the point. We use Elixir to solve one class of problems which its excellent in doing (notably: real time distributed systems.) In this area, there is nothing even remotely close in terms of capability. So, we choose the runtime over the language (elixir is a pretty cool dynamic language though!). Also, due to the unique nature of Erlang/Elixir, a lot of problems that exist in C/C++/Rust/Go don't exist in that language. Of course this has meant writing a lot more tests to test things that a typechecker should be able to catch. But it is what it is. We're following Gleam though to see where that ends up.

BTW, here's some places I've spoken about Discord's Elixir usage:

- a podcast: https://smartlogic.io/podcast/elixir-wizards/s5e8-nowack-hei...

- interview with José Valim (elixir's creator): https://elixir-lang.org/blog/2020/10/08/real-time-communicat...

Gleam looks like a very promising language! I hadn't seen it before.

https://gleam.run/book/tour/try.html

It also looks like there are currently some serious limitations:

https://github.com/gleam-lang/otp

I think a difference is that Elixir is deliberately dynamically typed, so you don't expect type safety. You write the happy path and rely on "fail fast" and the actor hierarchies to write robust code.

Besides, Elixir/Erlang has the best dynamically-typed equivalent of algebraic types (tuples + atoms + pattern matching) :-)

I agree, rust is so much easier once you've passed that learning curve.

Not having to worry if this random reference you're using has been overwritten somewhere else in the code is absolutely refreshing.

It allows me to bring back the mental map of my code and the variables to the exact scope I'm writing in.

This is an underrated feature of rust. Especially for newer devs who do not have that mental map of the code yet, and are worried that any small change they make might cause cascading failures. That's just something you don't think about in Rust.

Those reasons make much more sense than the GC focus of the blogpost. Definitely a lot of nice features in Rust.

The GC was apparently the reason behind rewriting that particular service. No wonder there are more reasons when you look at all the services across the company.

But you couldn’t help yourself even after being corrected by someone who knows what they’re talking about.

> rust is far easier

That's an unusual claim :) Could you elaborate? How is the learning curve, especially for fresh graduates who have been trained only on languages like C, Java and Python?

> The fact that you can just drop a future to cancel it, as opposed to having to thread around a ctx which may or may not work has been amazing.

Agreed. I like Go, but that's big drawback of goroutines and threads.

In my experience, Rust puts many costs upfront, which makes it harder to do stuff when you aren't used to doing it the Rust way, but makes it easier to get it right if you are used to Rust. It basically gives you a relatively strong guarantee that if your code compiles and has no warnings, it's mostly collect - yes, that includes logic errors to some extent thanks to the type system. That also makes refactoring easier - chances are, the bugs you possibly added by refactoring will get caught by the compiler. Obviously, that's not a complete replacement for testing, but it does reduce the amount of tests you need to write to ensure correctness.

Frankly, I don't consider myself a particularly good programmer - but when I use Rust, I feel confident because of being backed by the compiler. That's something I can't say about any other language, and I only have less than a year of Rust experience compared to 5 years of total experience programming.

Imo Rust is easier to master than other languages, but it's definitely harder to do basic stuff in it because of the upfront costs I mentioned.

A language being easy to write has no relation at all with it being easy to learn.

How would a fresh graduate be able to write safe C code when even experienced people are unable to?

I have worked with multi-threaded C++ code long enough to see that it's impossible to work in a team while keeping the code safe. Usually servers crash and canaries catch it in production, but if we have type system improvements that solve these problem, why not take advantage of it?

I've noticed this kind of reply on rust threads often. Someone says something like:

> Rust moves many potential program problems from runtime to compile time; it has a high bar to meet, but once you meet it the program is rock solid

And then somebody replies with:

> Oh my favorite language doesn't make me do all that work up front, therefore easier (TM). Why Rust so hard?

But the reply misses the point: Make no mistake, you're still paying the cost (with FASTCASHNOW-interest rates), but the cost is just smeared out over days or weeks as you fix, debug, and tune it during use. You may choose whether to do that work up front or while the program is in active use, but avoiding the work is not a choice.

> That's an unusual claim :) Could you elaborate? How is the learning curve, especially for fresh graduates who have been trained only on languages like C, Java and Python?

One way to look at it would be that Rust forces you to conform to a specific ownership model, and makes you be explicit about resource management and how things are shared and accessed. Having to do all that can make certain patterns awkward or hard to write, especially for those new to Rust who haven't internalized what the compiler wants yet.

But on the other hand because so much is explicitly spelled out it can end up reducing the amount of things you need to consider or keep track of while reasoning about code, making it easier. Especially for code which is essentially bolting libraries together with a bit of data transformation.

> That's an unusual claim

It's really... not. For people that have actually written big things in Rust. We keep repeating it, but we try to cautiously, tread carefully, lest we be accused of being part of this dreaded Evangelism Strike Force we keep hearing so much about.

Ease of use != ease of learning.

Rust is hard to learn (unless you have a deep ML/Haskell + C background). But once you're up the curve, it's a very expressive + productive language. Refactoring and maintaining large projects confidently is a breeze, and creating powerful abstractions that can be safely leveraged by large teams of programmers... just works.

When I read descriptions like yours, Rust sounds like the language delivering on the promises of Haskell in the Real World.

Some promises of Haskell.

Rust is still a very low level language, but if you like Haskell, you won't go into anything else when writing some low level program.

Haskell overpromised a lot.

When I looked at it, the favourite quicksort example was so elegant, I wanted to learn the language. Then after I started looking at the run time (after wasting a lot of time going through learning Haskell), I realized that it was all a lie, ant that sort wasn't quick at all.

I don't see the same overpromising problems with Rust: you usually get the performance that you think you will get when you're writing the code.

So let me get this straight: You don't like Haskell because someone used it to write an inefficient quicksort example online?

Not somebody, it was on the home page as a use case of how much better Haskell is than other languages.

The default libraries are extremely inefficient.

Generally if I use Python/Ruby I can get a 50x slow down that is accepted, but with Haskell it can be more, as it's very hard to think about what the compiler does (memory usage can blow up easily).

To be fair, that code is to illustrate the beauty of the language rather than efficiency.

(Also the fibonacci sequence using 'zip'. Blew my mind.)

Haskell folks are fairly upfront about possible performance issues. Lazy evaluation comes with its own set of tradeoffs.

You can write very inefficient code in C++ if your algorithm isn't efficient.

I still don’t see any tradeoffs explained, quite the opposite is on the home page:

,,The purity of Haskell code makes it easy to fuse chains of functions together, allowing for performance benefits.’’

Having something ,,beautiful’’ and inefficient is the problem I’m writing about. The language itself is guiding people to inefficiency.

I haven’t felt this with C++, so I’m not sure what you are talking about: usually simple / pretty code runs fast there, and more complex code runs slower.

Yeah, Haskell won't give you Rust-like performance. Very few languages will and, except for C, you won't even see them here on the comments.

Haskell can have a reasonable high performance, but if it's you main concern, or if predictability is important, you need a low level language.

Right now I'm using Julia for high performance math heavy modelling with lots of experimentation, and it's great for this purpose. I can get most of C++'s advantages without giving up on flexibility. My code base wouldn't scale to team work / production software though as well as C++.

> Often as it is in this case with no gain for the company, other than some resume building for the involved team members

This is too cynical. A large re-write will not be approved at any large companies without millions of meetings and documentations filled with justifications.

... All of which are researched with proof of concepts. Followed by some small trials. And finalized with one or two actual rewrites of non-critical software.

I'm not saying that all companies do it this way, and that there is no "resume building" or "magpie-ing" by some engineers in many companies. But it is perfectly possible to state a thesis "Rust is better for our X and Y and probably for Z". and to then research and prove that thesis.

> resume building

We call this project farming where I am. I'm not a dev and don't work in IT industry, but I do work for a large company and every few months some new project is rolled out to 'improve' things and most of the time we all collectively groan and wait for the person to get promoted.

The most frustrating part is when once in a blue moon the project turns out to be useful, and the person gets promoted off of it and abandons the project. Nobody then wants to pick it up since they won't get any credit for running it and it ends up getting abandoned.

> Edit: On the other hand, Discord could have made the very valid decision that they have an easier time recruiting for Rust than for Go. Which is in fact a good reason to switch.

In other words, discard the technical arguments laid out in the article and just speculate that it was an HR/management decision.

Reply on news.ycombinator.com