Erlang Garbage Collector | Hacker News Reader

Erlang Garbage Collector

erlang-solutions.com

87 pointssigna1127 comments

> Each Erlang process has its own stack and heap

This is one of the cool things about the Erlang VM. Each process (lightweight thread) has its own heap! In other VMs threads can affect each other by generating garbage and causing GC pauses. In Erlang even the heaps are isolated.

How valuable is the property that other threads' GCs won't stop your thread if your thread bears its own full stop? It seems like a concurrent collector with a very small STW phase would be preferable. Lastly, the lack of shared memory certainly enables simpler GC schemes like this one, and it precludes all sorts nefarious bugs, but it also makes efficient parallelism impossible in many cases. Depending on your application, that may not be suitable.

These are good questions.

Concurrent compacting collectors are pretty tough however, and a single process STW should be much faster than a global one, since it will be (at worst) proportional in time to the per process heap size.

I suppose it would need to be compacting; Go seems to get along okay without compaction, presumably because it has value types?

Go trades the complexity of a compacting GC for forcing you to restart the app regularly if you have a workload with plenty of heap allocation.

Really? I've never heard anyone mention this and I've followed Go for years.

Yeah, it mostly happens when you have a server which runs an abnormally large task or payload, causes the heap to grow, and does it in a way that Go can't release the memory back to the OS due to heap fragmentation.

I haven't looked at this for a while but I believe it's still an issue.

https://github.com/cockroachdb/cockroach/issues/2007 https://github.com/golang/go/issues/18896 https://github.com/golang/go/issues/14045

Go has traded throughput for latency. The throughput of a semispace copying collector is very high for most workloads, but can suffer latency issues. There is literally zero cost to collecting garbage in the nursery with the algorithm described in TFA (a nursery collection has a cost proportional only to the live data), and the cost of allocating in the nursery is just incrementing a single value. However eventually you need to collect the entire heap, and it's still proportional to the size of the live data, which is usually the majority of the heap (since you've already collected the nursery at that point).

Sorry, I'm confused. Is Go's GC the semispaced copying collector or the TFA? Or neither? Or both?

TFA == "The Fine Article" so refers specifically to the Erlang Garbage Collector.

"Semispace" is one term to refer to the family of garbage collectors that Erlang's derives from. The heap is divided into two spaces, hence "semispace"

So Go's GC is neither. Go's gc is tuned for lower latencies, and that tends to rule out moving collectors (there are ways of having incremental moving collectors, but they complicate the runtime a bit).

Ah, thanks for the information and clarification!

Even incremental GC is easier to write if you only have one thread you need to collect for.

On Erlang 20 long GCs run on the thread pool for “dirty” (blocking) tasks, they don’t block a scheduler, so the real-time properties are nice.

There have been implementations of Erlang with shared heap in the past, before BEAM became heavily multi-threaded: https://www.researchgate.net/publication/2371933_A_Case_for_...

I think it would be nice to see this tried again. Perhaps the issue is scaling to future systems with potentially hundreds of cores?

The heaps are smaller. The problems with gcs start when you have multi gigabyte heaps. If every heap is only a gigabyte it will take at worst 10ms per collection. (Anecdote) If that is too large then you split your heaps into smaller ones until the pauses are acceptable.

Also remember malloc isn't free either it takes time to execute.

> How valuable is the property that other threads' GCs won't stop your thread if your thread bears its own full stop?

GC on a small heap is fast; especially since the language does not allow for circular references in the heap. Almost always, if you're you've got a large heap in your Erlang process, you're doing it wrong; large data generally belongs somewhere else like ETS, the in memory term storage.

Last year's post updated for Erlang/OTP 20. This is the number one information source on Erlang's garbage collection. Well written.

Can someone more familiar with Erlang/Elixir say some drawbacks of BEAM? I'm learning Elixir now and as I become more familiar it seems very difficult to write buggy code.

If only it were typed, ah... (no, dialyzer doesn't count).

Drawbacks, well it's slower than other language ( Java, C#, Go ), it's not typed, the deployment / process liveness mechanisms are now useless with Kubernates / Docker that are better at doing that and language agnostic.

It can be slow, or not. We wrote some serialization code in Elixir and Java, and the Elixir version was a little faster. It's good at that.

On the other hand, I had some bit twiddling code that was 13x slower than C. I know because I rewrote that section into C as a NIF and benchmarked. The nice thing then was my overall 95% Elixir app became about as fast as a 100% C app.

Exactly, beam is a pretty slow and unoptimized VM, CPU wise.

Which is not so important, as their strength is eliminating IO waits and context switches which are usually much costlier.

Their GC is top notch.

Can you expand on the liveness mechanism bit? Even if K8 could automatically restart containers, I feel without the language itself inherently being able to catch such things, it's not as useful as BEAM processes.

Comparing to erlang, docker is pain in the ass. You need to expose one more port to your system? Restart at least one cotainer. Need to link additional containers? Restart at least one.

In erlang you just execute some erlang commands and in most cases you don't even drop connections, it's like changing your cylinders in engine when it's still running. Of course if your code has errors it can go wrong, but typically you are only spammed with some errors from this one service until you fix the error and reload code.

After you do some code upgrades in erlang, you miss this functionality in all other languages.

How would you implement types? receive / send make dealing with types incredibly difficult. Also, hot code reloading makes reasoning about types difficult as well.

There's been some proposals for Multi-party session types, but none that are practical by any means.

Erlang process are inherently stateful, which makes it hard to fit it into modern deployment infrastructure such as kubernetes that works best with a stateless process.

The process model also makes it very tempting to build a monolithic-microservices and build an erlang VM cluster which can add complexity if you already have an infrastructure like kubernetes or works with other languages.

FWIW, these things I just mentioned can also be considered as Erlang/Elixir's strengths depending on your requirements.

It's typed... Dynamically. You mean static types right?

When BEAM crashes (which is rare), it turns out your isolated processes aren't isolated. You could mitigate for this by running multiple BEAMs, but that comes at a cost too.

BEAM works best if you can model your work as a process, optionally with state, getting messages and sending messages. If that doesn't fit your program, it's not going to be a good fit.

There's not a long of bindings into other systems; GUI bindings are problematic -- OTP ships with wxWidgets, but it doesn't feel like Erlang programming to use; I didn't evaluate gtknode, but last commit 2 years ago seems iffy for a project addressing gtk (which I was under the impression changes frequently). If you're dealing with networky things, and the protocol is well documented, it's easy to create your own bindings; it's possible to create your own bindings to other things, but dealing with Erlang data types in C is some amount of effort.

Hotloading will warp your mind. For a side project I had written in C, I wanted to hotload the main loop so I didn't need to lose state when I restarted (serializing the state was out of the question because I'm keeping state in a library that doesn't offer serialization); I looked into writing bindings for the interfaces I needed, but it was going to take too much work, so I ended up refactoring my code so I could use dlopen to hot load the main loop in C. :D

There are some issues I've run into with OTP, but those aren't really limitations of the BEAM. Lots of OTP doesn't impose explicit limits (or have ways to impose limits), but instead has scaling issues when you hit large numbers / exciting conditions. Most of these _are_ easy to fix, but some of them are frustrating, and are clearly only still broken because there's not a huge community of people, some of which would run into things. For example, the virtual binary heap that was mentioned in the article -- this is a great solution to the problem (the problem is: if you have a simple process that just passes around binaries, it doesn't generate enough garbage on its heap to trigger GC, so the shared binary heap grows rather large), but the fix is pretty recent because not many people ran into it, and when they ran into it, they tended to fix it through manual GC rather than fixing the GC.

Please, those who control this site, if you're reading - don't use medium grey text with very light font weight on a a white background.

Reply on news.ycombinator.com