Erlang programmer’s view on Curiosity Rover software
jlouisramblings.blogspot.com
jlouisramblings.blogspot.com
But the article implies (and more than once) that the rover's architecture borrows from Erlang, while the opposite is true. Erlang adopted common best practices from fault-tolerant, mission-critical software, and packaged them in a language and runtime that make deviating from those principles difficult.
The rover's software shows Erlang's roots, not its legacy.
The key take away is that these methods seems to work. They worked in a setting of C code on the existing rovers as well as the newest one. They worked on large telephony switches written in Erlang. My guess would be that many other mission critical systems will posses the same traits.
But what I like most about Erlang (and I like Clojure for the same reason) is that it is not a kitchen-sink aggregation of programming language features thought to be useful or cool (like some other languages that I won't mention), but a perfect (or near-perfect) mix of just the right features, all made to work with one another in synergy.
In other words, Erlang, like Clojure, does not say, "here are the features you can program with", but rather, "this is how you should program". And the features these two languages provide are, I think, just the right ones for developing software with modern requirements (scaling and high-availability) on modern hardware (concurrency).
Erlang's features, however, do tend to focus more on programming in the large, or high-level software organization (they helps making your software scalable and fault-tolerant) and less on low-level constructs (it's hard or impossible to write a super high-performance data structure in Erlang). It delegates those low-level problems to units written in other languages. Clojure is the opposite, I think. It's designed to make building correct and efficient data structures and data-processing functions easy, but provides little guidance to high-level organization of complex, large software.
Both are also relatively easy to learn, which only shows their elegance (though Erlang does suffer from a somewhat antiquated syntax owing its origin to Prolog).
[1] http://www.erlang.org/doc/man/ets.html
[2] http://www.javacodegeeks.com/2011/04/erlang-vs-java-memory-a...
There are versions of the JVM for hard real-time applications (Erlang is for soft real-time only), that provide explicit memory management, and a very fine-tuned GC, like this one: http://java.sun.com/javase/technologies/realtime/index.jsp
I know of some defense applications that use it.
This process (which Ed Jourdan neatly eviscerated when applied to business software) produces software that is as reliable as the specification and underlying hardware.
[1] Software had been there for a few years.
Any _robust_ C program contains an ad-hoc,
informally-specified, bug-ridden, slow
implementation of half of Erlang...
On another note, it's pretty cool that the first three names credited in the JPL coding standard document (which is linked to at the bottom of the OP and is surprisingly well written) are Brian Kernighan, Dennis Ritchie, and Doug McIlroy.
On the second line of linked article:
"This is a humorous observation"
(emphasis mine :) )
Have you seen the rover move? :) http://www.youtube.com/watch?v=484R2TQGLAQ
Jokes aside, there isn't much of a speed requirement here, really. It all runs fine on a radiation-hardened 200mhz processor. Nothing on mars is running away.
It's not surprising at all. It's the work of Gerard Holzmann[1], who worked at Bell Labs at the Computing Sciences Research Center with all the other Unix folks where he developed the Spin[2] model checker. (As a sidenote, Spin was used to check the sanity of the Plan 9 kernel).
Best "Smug Programmer" line ever.
Memory limits would be great to give you some ease of mind when reusing libraries for processing input. For example processing XML, even using SAX parser, still leaves you open for attacks like huge tag values or just something more real world, huge href="data:" attributes. Obviously the right way is to have a limit in the parser (which xmerl doesn’t have, unfortunately), in this case, but it’d be nice to have some safety net from the VM as well.
Edit: in response to the reply, I suppose I should have mean tasks instead of "processes" (which in VxWorks would be the RTP)
[1] As someone mentioned, VxVorks 6 did introduce processes and "usermode", called RTP. As with most features of VxWorks, you compile that into your image if you want the feature. But there's a lot of inertia, and much of the VxWorks stuff I see doesn't use RTP yet.
I know some clever folks in the business have done interesting work on ML-to-C compilers, but it's still in the early R&D phase at this point - the compiler itself would have to be thoroughly vetted.
That leaves you with very few languages you can use, and C is a good predictable one for the problem. Its tool support is also quite good with static verification etc. And it is a good target for compilation. As someone else notes, most of those 2.5 Megalines are auto-generated.
"TL;DR - Some of the traits of the Curiosity Rovers software closely resembles the architecture of Erlang. Are these traits basic for writing robust software?"
Contrast this with https://www.ohloh.net/p/erlang: "In a Nutshell, Erlang has had 7,332 commits made by 162 contributors representing 2,346,438 lines of code"
I'm not sure if those roughly 154kloc really make a difference...
I’m not really sure what point are you trying to make…?
On the other hand, every Hello World in Erlang drags in about 2.5MLOC of liability (even if much of that is never run). And I doubt it's all autogenerated.
So if anything, 2.5MLOC of generated NASA code is probably less complex than the erlang runtime.
'immutable' and 'shared' are added to the known C 'const' qualifier for data that will never change (contrary to not changing in the declaring scope only) data which is shared across threads, everything else is encouraged to use MPI using the std.concurrency module.
Pure functional code can be enforced by the compiler by using the 'pure' qualifier. There is even compile time function evaluation if called with constant arguments, which is awesome when used with it's type-safe generic and meta-programming.
There's unit tests, contracts, invariants and documentation support right in the language. Plus the compiler does profiling and code coverage.
I'd be curious to test D against Erlang for such a system. (Not saying Erlang shouldn't be used, it's the next language on my to-learn list, just that the switch to functional might be too radical for most developers used to imperative and OO and D provides the best of both worlds.)
This is now past history as the community has joined around D2, just known as D, and strives to reach compliancy with the "The D Programming Language" book written by Andrei Alexandrescu.
D2 development is made in the open with source code available in GitHub.
Besides the reference compiler, dmd owned by Digital Mars, there are also LDC and GDC compilers available. Currently it seems that GDC might be integrated into GCC as of 4.8 release.
Right now it seems more people are picking up D, mainly for game development projects.
Also, what compiler does NASA use? Something like CompCert? What kind of compiler flags? Do they run it through an optimizer at all?
However: I'm dubious that it's a strength many people here need. No, the article did not say anything about that, but I am. A few minutes of downtime, now and then, for a web site that's small and iterating rapidly to find a good market fit, is not the biggest problem. And while Erlang isn't bad at that, I don't think it's as fast as something like Rails to code in, and have all kinds of stuff ready to go out of the box.
That said, I'd still recommend learning the language, just because it's so cool how it works under the hood, and because sooner or later, something will take its best ideas and get popular, so having an idea how that kind of thing works will still be beneficial.
The author’s previous post has a good overview of how message passing contributes to that as well.
http://jlouisramblings.blogspot.com/2012/06/protocols-in-kin...
STM and MP are not a priori against each other. You can have both and sometimes one is better than the other for a certain kinds of problems.
> It turns out that many of the traits of Erlang systems overlap with that of the Rovers. But I don't think this is a coincidence. The software has certain different properties — the rovers are hard realtime whereas the erlang systems are soft realtime.
> The things that do not overlap has to do with the need of having soft realtime vs hard realtime. In Erlang we can yield service. It is bad, but we can do it. On a rover it can be disastrous. Especially in the flight control software. Fire a rocket too late and you are in trouble.
But the concepts are very similar - message passing being the way to communicate between modules, rather than shared memory ways.
This brings another topic - the Linux vs Minix debate :) - I guess there are right things to be done for the right time, and right target. It's just getting all these things right is the hardest.
Only BEAM is an interpreter, there are HiPE and ErlLLVM backends as well. You can also write NIFs — functions in C that can be executed within VM.
Deleted comment
vibe.d on the other hand is cooperative since it uses coroutines for concurrency.
My understanding (coming from C and OS terms) is that pre-emptive means taken over. e.g. if I have a real OS thread it is being temporarily "paused" and the resources (CPU/mem/io) are given to something else. At some point control is restored back.
But this is without the knowledge or instructions from the thread itself. So things like priority inversions are hard to battle with pre-emptive multitasking - for example thread A with low priority holding a mutex, while thread B with higher priority waits for it. (and no need for mutexes, if only message passing is to be used).
What seems like cooperation in node.js is really just async operations queuing up on the event loop. Since requests are also async events, they get interlocked with callbacks from existing requests.
To me, cooperation is when you yield the thread to another coroutine. This saves the state of the call stack, the registers, everything; meaning you don't force your user to keep that state in closures. The user code in a cooperative environment feels sequential and blocking and results are passed by return values, not by calling continuations.
Its also friendlier to exceptions since it doesn't lose the entire call stack; with node.js you only get the stack since the beginning of the current event loop's tick.
There’s a single loop which blocks until the task yields while waiting on the result from one of the worker threads. That’s cooperation. All queued connections are starved until that happens. In Erlang, or just with pthreads, the connections are processed independently. Think separate event loops for each connection.
> To me, cooperation is when you yield the thread to another coroutine. This saves the state of the call stack, the registers, everything; meaning you don't force your user to keep that state in closures. The user code in a cooperative environment feels sequential and blocking and results are passed by return values, not by calling continuations.
That has noting to do with how execution is scheduled. Cooperative scheduling requires passing continuations[1], so the execution can be resumed while it waits. The simplest implementation is to use callbacks, the way node.js does it. Futures and deferreds[2] are a little bit more sophisticated (Python’s Twisted, probably something for node.js exists as well), as they allow for better composition. And of course you can hide the continuations entirely, which can be done in both Scala (compiler plugin) and Python (gevent or using generators), rewriting the direct control flow by breaking it on yield points automatically (this is how exception throws work in most languages btw), but the limitations inherent in having a single event loop per thread will still exist.
[1] https://en.wikipedia.org/wiki/Continuation-passing_style
Yes, node.js is cooperative, yet since all I/O is asynchronous the time spent blocking is mostly dispatching and simple operations, it doesn't block while waiting - that's where it's performance and high concurrency comes from. Doing CPU-heavy work in the server/main process is a no-no.
And it doesn't have message-passing since it doesn't have multiple processes. There's nothing to pass messages to.
The only design pattern it shares is splitting code into subsets, and every language (that matters) has that.
(and I feel stupid having to apologize for a harmless comment. way to go HN)