Solving the Go Challenge 1 in Erlang | Hacker News Reader

Solving the Go Challenge 1 in Erlang

medium.com

177 pointsdavidw42 comments

Funny, when I attempted this Go Challenge #1 I had the exact same reaction, and halfway through ended up doing a solution in Elixir (which compiles to the Erlang VM) before finishing the Go one.

I think the the particular problem (which by the way was super fun, great work @mattetti!) was really well suited for pattern matching, so like OP I gravitated towards those tools.

I had been keeping my solution private until the contest was over on the 28th, but for sake of comparison of Elixir/Erlang idiomatic code (which should be pretty similar), I'll make my repo public now too (which also contains my Go solution):

https://github.com/mroth/golang-challenge-1

BTW, if you enjoy playing around with reverse engineering, I recommend NOT looking at any of these solutions, and doing the challenge yourself first (in whatever language you want). It was amongst the best of the "code challenge exercises" I've attempted.

Erlang pattern matching over binary data is one of the greatest things about the language. I basically copied the idea wholesale for ocaml-bitstring[1].

[1] https://code.google.com/p/bitstring/

I assume you are aware, but just to be sure: Friendly reminder that google code is going away and you should migrate this somewhere else ;)

I know I know :-(

Exactly. Almost seems like cheating, but in a good way.

You want your tools to feel like that.

Go challenge author here, really glad to see this post and a great read for Go and not Go developers.

Author here. I'll try to watch this and answer questions.

Is there a link to just the code somewhere?

Now there is:

https://github.com/jlouis/go-challenge-1

If you need a LICENSE, I can slam a MIT or Apache 2.0 on it for you, but...

Erlang's processes are just threads implemented to act like processes is this correct?

Erlang's processes are not native threads or native processes. In a sense, they are "green processes" multiplexed M:N on native threads.

Basically they're processes within the VM that the VM manipulate the native threads?

Are there any good books on this subject?

Thanks.

Depending on how deep you want to go:

http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-...

BTW, this is the author of the program and the article!

Heck checkout out all his posts, they are fantastic.

Here is my quick try at it:

Try to think of Erlang VM as a mini operating system. Erlang processes are then OS processes.

If your operating system is modern it lets you launch more processes than you have CPUs.

Also if one process crashes it doesn't affect others. This is pretty cool and useful. Anyone remember how much fun it was to run Windows 3.1 or Windows 95? Your game crashes and your word processor got screwed as well as a side effect. That is pretty primitive. We got better tools than that (Linux, Windows, etc). So this takes you a long way as far as the analogy works. Erlang processes have the same properties.

The only difference is that while you can probably have hundreds or maybe thousands OS processes on a large system, you can have many hundreds of thousands or even millions of Erlang processes. They are really lightweight (a couple of KB of memory only when they start up). The VM is a really awesome piece of engineering because it makes it all possible while also preserving isolated heaps (memory).

Erlang and OTP in action is my favorite, as it cuts to the parts that I as a distributed systems engineer really wanted to learn. http://learnyousomeerlang.com/ is one of the most commonly recommended guides/books.

http://learnyousomeerlang.com/ is excellent, and you can read it online or buy a paper version.

Here is a Julia version for anyone who is interested (print's to stdout instead of passing a struct though).

https://gist.github.com/anonymous/23bbdf1f5077c7669844

I wish there was more explanation with the = sign in Erlang. In that it's not an assignment but a pattern match first.

IIRC, it pattern match first and if the variable is not assign then it'll assign it IIRC. The code and explanation just state that it's Erlang to do assertion.

I think that's an interesting concept at least when I play with Erlang.

I guess one interpretation of Erlang's = (this is from someone rather new to the language) is "Make the thing on the left equal to the thing on the right without mutating existing variables. If that's impossible, error". That nicely accounts for the apparent dual nature of the = operator.

I don't know Erlang, but I have done some Prolog and this sounds an awful lot like unification of an equality predicate.

Erlang has some roots in Prolog. Its initial version was actually implemented in Prolog.

It is unification to a certain extent, but unification is slightly different IIRC the semantics correctly.

I think I just fell in love with Erlang.

Erlang is a wonderfully pragmatic and mature language, it doesn't get nearly the hype it deserves.

It's let down by being more of a ”throw it over the wall” project than an open and transparent one. (There is no public bug tracker, road map, etc.)

What do you mean throw it over the wall? It has been solid and running large, reliable, industrial system for many years. What do you want Ericsson to bake a cake for you? It is open source (https://github.com/erlang/otp), some have forked it and updated as needed (for example WhatsApp). There is a bug mailing list (http://erlang.org/pipermail/erlang-bugs/), there are regular bug fixes, pull requests, frequent releases. New features added (like maps).

> What do you mean throw it over the wall?

It's more closed than it is open.

> It has been solid and running large, reliable, industrial system for many years.

Did I say otherwise?

> What do you want Ericsson to bake a cake for you?

Yes.

> It is open source (https://github.com/erlang/otp), some have forked it and updated as needed (for example WhatsApp).

I know it is open source. I have contributed to it.

> There is a bug mailing list (http://erlang.org/pipermail/erlang-bugs/),

Presumably you bring this up because I pointed out there is not a public bug tracker. A mailing list is not a substitue for a bug tracker.

> there are regular bug fixes, pull requests, frequent releases. New features added (like maps).

Yes, and? Where do I go to see the status of, I don't know, native processes? (Assuming that's still being worked on.) How about a list of bugs in the ssh or ssl apps? (This is not an esoteric thing: http://erlang.org/pipermail/erlang-questions/2015-March/0837...)

Lastly, I'd ask you to consider what your response has done to improve and grow the Erlang community.

> Did I say otherwise?

Yes you did, you said "It's let me down". I just assumed t was something more serious like major design flaw, or periodic segfaults in the runtime, and not "bugs go in the mailing list" vs "no Github issues".

> > What do you want Ericsson to bake a cake for you?

> Yes.

And they will. Throw in a couple of millions their way in there and they'll take a few engineers off from building their product and you might get a delicious cake.

> > There is a bug mailing list (http://erlang.org/pipermail/erlang-bugs/),

> Presumably you bring this up because I pointed out there is not a public bug tracker. A mailing list is not a substitue for a bug tracker.

It is not but they are the ones who pay their developers to work on this and they have a bug mailing list. You can still issue pull requests if you want. There are plenty here:

https://github.com/erlang/otp/pulls

File the bug, submit a patch and discuss there. Some pull requests have 20+ comments.

> there are regular bug fixes, pull requests, frequent releases. New features added (like maps).

Yes, and? And it is not exactly a "throw over the wall. It is like getting a free car from someone but complaining that the seats are the wrong color.And the car has now failed you.

> Where do I go to see the status of, I don't know, native processes?

Ask on the mailing list. Comment on the pull request or go on #erlang on IRC.

> Lastly, I'd ask you to consider what your response has done to improve and grow the Erlang community.

But what has your response done to hurt it? Here someone shared a cool project, and someone else commented how cool it is. And you are complaining that it is a "throw over the all" and it is a failure because they didn't set up Github issues properly. If you are going to spread fud then, well, be prepared to get nasty comments back.

Wow. What a mountain from a molehill. Your response has so little to do with what I've actually said that I can't see any useful way of responding. Good day.

I have one question. It is often said that in Erlang, one can have a large number of threads. However, doesn't that cause problems with the memory hierarchy (cache)?

Erlang runs about K threads where K is the number of cores towards the OS. All processes are "logical" in the same memory space and usually they happen to be pretty small with a heap of a few kilobytes. As long as this is the case, then even swapping among a number of threads, say 100, will not take you out of the 2nd level cache.

If you have a large number of processes, then it is a problem. And do note that Erlang programs have way more pointer-redirections than a typical array-like approach which is common in Go and other imperative languages. That is the memory pressure by default will be higher than what you can achieve in languages with static types.

It tends to be a problem for tight computational kernels, and Erlang programmers usually fall down to C if they prove to become bottlenecks.

I have an older post here describing the schedulers: http://jlouisramblings.blogspot.dk/2013/01/how-erlang-does-s... which is still reasonably up-to-date to give the overview.

You can have hundreds of thousands of processes easily in Erlang. An Erlang process is very lightweight (only a few KB of memory!), yet, just like a real OS process it doesn't share its heap (memory) with the other ones. So if process 100000 crashes it won't affect the other 99999 left around.

I tried rust on this. https://github.com/tuxychandru/golang-challenge-rust/tree/ma...

It was shorter in LOC than my go version, mainly due to try!, but the lines themselves were longer. The one difference in correctness was that I forgot to close the files in Go, but RAII took care of it automatically in Rust.

I'm primarily a Python programmer, but have had my eye on Erlang and Rust as other languages of interest to look into at some point. While those two are to my knowledge meant to solve problems in different domains, which one do you think would be easier to grok or get going with faster between the two? I have no C background. Just Python and SQL. I realize easier to get going with is a poor metric. It's not the only one important to me but I'm just curious. My initial looks into these led me to believe Erlang would probably be the easier of the two coming from Python.

I love the looks of Rust because it's C ABI compatible and I could write Python libraries with it.

Erlang because it looks to be the best thing going for webservices, which is much of my work.

I'm not quite sure there is that much overlap between Erlang/Elixir niche and Rust niche. You can certainly do "most" things with both but they will shine in different circumstances. You can certainly build anything with Rust, even an OS. And I guess this is where it will shine: system development, and very fast applications/tools.

Erlang/Elixir is made to build fault-tolerant applications: think a service (which is really tens of thousands of micro-services all running in the Erlang VM), distributed on a cluster, that needs to run 24/7 without any interruption ever. Upgrading the app should not halt the system. A bug somewhere should not require you to restart the app: it should be limited in scope and you just hot patch it away. Erlang is fault tolerance is fault tolerance.

From a Python/SQL background both will have a different set of challenges to learn: Erlang/Elixir means learning functional programming (no for loops, all map/reduce/filter and recursion), while Rust means learning how to handle references/memory ownership.

To build webservices I'd give an edge to Elixir: cowboy + plug (+ phoenix). But that should be fun to build with Rust too... You should learn both :)

Rust guy here, and I can tell you that there is a huge contingent of our community made up of dynamic language programmers who have been seeking experience in systems programming. That said, when it comes to systems programming understanding the system itself is just as crucial as understanding the language, and that takes time. If getting up to speed quickly is important to you, perhaps you'd like Elixir ( http://elixir-lang.org/ ), which is sort of a Ruby-esque language that runs on the Erlang VM.

Thanks. I've looked into Elixir, I like it, but decided to not look into it further. Erlang is already niche enough as it is.

I got stumped decoding the tempo. I see in a spoiler that the Hex is 00 00 F0 42 but I don't understand how 120.000 is decoded from that.

    > state( "Magic",       { consume(16), toString(),  assert("SPLICE") } ),
    > state( "Length",      { consume( 1), fromOrd(),   cropInput() } ),
    > state( "Version",     { consume(32), toString(),  setVersion() } ),
    > state( "Tempo",       { consume( 4), fromFloat(), setTempo() }),
    > state( "Instruments", { exhaust([snip]) } ),

The layout, from my abortive attempt to write a parser.

http://en.wikipedia.org/wiki/Single-precision_floating-point...

Look up how we usually encode IEEE 754 floating point numbers and then also think about endianess (little or big).

Thank you. I feel better prepared to lookup binary formats now. I should have played with Synalize a bit more.

I got stuck on what TFA calls Len. I didn't need it to solve the problem but I couldn't have modified the file without knowing what it was. Luckily a friend of mine saw it almost right away.

A fun problem!

My first version didn't account for the Len field either. It was only after a while someone told me about its purpose via a hint.

Reply on news.ycombinator.com