MLton with Realtime GC and Threading Features | Hacker News Reader

MLton with Realtime GC and Threading Features

github.com

127 pointseatonphil36 comments

Since the repo and group are a little light on details, here is a paper by the group describing some of their work [0].

As always, if you are interested in learning OCaml or Haskell, try SML instead! You can find a defense of this thought here [0] (tldr; it's a simpler language that makes it easiest to learn the foreign concepts that OCaml/Haskell/Scala employ mercilessly). Of course, if you actually want to get something done I'd recommend Haskell or Scala.

If you're interested in SML, I recommend checking out the /r/sml wiki [2]. The Stack Overflow community is very helpful and personally I hang out on #sml on Freenode.

[0] http://sigbed.seas.upenn.edu/archives/2016-04/d1.pdf

[1] http://ponyo.org/news/ponyo-for-standard-ml

[2] https://www.reddit.com/r/sml/wiki/index

And if you're working on web apps, look at Ur/Web[1] which also uses MLton. It is what Opa should have been and has a very nice model.

[1] https://air.mozilla.org/ur-web-a-simple-model-for-programmin...

Edit: Ur/Web takes inspiration from Haskell, SML and OCaml, and is arguably an improved substrate of the good features of those. It's definitely the nicest tool I know of for writing web apps at the moment.

I'd agree, but Ur/Web gets more difficult when you consider that it's extremely CRUD focused due to the way page handling and 'memory regions' in the implementation work, and how deeply ingrained this is with its transactional view of the world. It doesn't even have a garbage collector. Shoehorning anything 'non-transactional' into the `transaction` monad (like sending email, writing to disk or firing the nukes) can range from achievable to very difficult to "fuck it, just use the database for everything" or whatever.

To be fair, "fuck it, just use the database" is a totally fine solution in many cases.

And yes, I agree Ur/Web is amazing given what it does. The edges are very rough (I don't know if it's the nicest tool by every metric, but definitely several of them). But it's worth considering if you have a mature level of FP experience.

You can also compile Standard ML to Javascript using SMLtoJS [0] via MLKit [1]. melsman even has an online IDE [1] with which you can play around with his reactive web stuff.

[0] http://www.smlserver.org/smltojs/

[1] https://github.com/melsman/mlkit

[2] https://smlserver.org/ide/

I really like SML, but still havn't seen a good unicode solution here - which lets me use unicode in string constants in code, as well as convert data between encodings at run time.

That's what's missing for me. So for now, F#.

Have you considered converting everything to utf-8 up front? (either in SML or in another language if it's a batch process) You don't need 16- or 32-bit wide types to handle unicode. Instead of an array of chars, utf-8 can be used directly in its encoded representation -- i.e. "char*" -- which basically every language is compatible with.

Almost every algorithm you need works fine in this manner. You rarely need O(1) access to a character like c=s[n]; usually you can just iterate forward, parsing the utf-8 stream (e.g. see Go's rune library).

https://blog.golang.org/strings

Now you have two problems: managing utf-8 encoding (the idea that you won't want constant access into a string is pretty laughable—do you not ever use ranges?), and trying to work with a language that doesn't support unicode first class.

Random access by code point is hardly ever quite what you want, and UTF-8 is not a bad internal encoding.

http://dparrol.github.io/unicode-random-indexing.html

Once you have a range, access to either side of it is constant time. The only thing you can't do is to "skip ahead X codepoints" in constant time. And I can't think of a single use case for it.

That is an excellent point. There are a few notable shortcomings in Standard ML and I often wonder what the harm would be to create a language that diverges from the standard to support some very basic (but fundamental) features that one needs in 2016. But then I remember that's similar to how Haskell happened and I shy away.

The SML standard defines a WideChar and WideString, that are compatible with the String type.

A SML implementation could just say 'WideChar' = UTF-32 and WideString is unicode, and all source files are utf-8. I think that would be within the standard.

My reading of the MLton page [0] on the subject suggests that the standard is in fact insufficient for full unicode support.

[0] http://mlton.org/Unicode

For those not familiar with static analysis, MLton is very famous due to http://mlton.org/WholeProgramOptimization

This looks really cool. Perhaps someone can explain to me why we need new languages, in the face of rich, powerful, languages/tools such as this. Is it just not-invented-here syndrome + better advertising? More forgiving to beginners? Corporate backing? Better/worse documentation/tooling?

I see new languages with features that don't seem significantly different from what already exits in languages with many years of development and optimization.

I wonder if instead of designing new languages, we should just choose ones with most of the features we want, and build out from there. It would save each team from rewriting (and debugging, hardening, optimizing) the wheel every tie.

"Perhaps someone can explain to me why we need new languages, in the face of rich, powerful, languages/tools such as this."

I became familiar with ML two years ago when I went back to college for a graduate program and had to take an undergrad pre-req. We used the SML/NJ compiler set. Immediately I loved it.

ML looks to be 43 years old according to the Wikipedia page. The answer to your question is a confluence of factors: paradigm shift, a greater need for efficiency years ago, and no corporate backer. In 1973, I think it was just too much of a departure from established programming patterns: think assembly, COBOL, LISP, and FORTRAN. The large computer technology companies were not willing to bet on it. Hence, fewer financial and human resources for fixing some warts people are pointing out in this comment thread.

When I was an undergrad around 1985, the grad CS students were crazed to get as much C experience as possible. Although I can't be certain of their motivation, I speculate that it allowed them to break from coding for insurance and banking companies using COBOL. C was simple to learn and made efficient use of the hardware: perfect for what was needed at the time with IT about to spread to medium and small companies.

In the 90's, OO became a big deal and we were looking for a language that allowed us to reduce the time it took to build against different hardware platforms. Java, with Sun as the parent and cheerleader, fit the bill.

Functional came into the spotlight maybe 10 years ago. (I'll give Scala a lot of credit for drawing developers into learning about the functional paradigm.) As a result of this paradigm's exposure, languages initially created to explore that space are coming into the spotlight. However, they haven't had the advantage of industrial financial backing for the past four decades, so warts are still there.

ML is a beautiful, powerful language. (I'd love to work with it as my daily language.) Had we adopted it instead of C or Java years ago, we would likely have not been familiar with off-by-one and null pointer exception errors. Certainly, people would be writing code much differently.

It's a shame, but we can only go forward. Maybe ML will have its day yet.

> It's a shame, but we can only go forward. Maybe ML will have its day yet.

Maybe, though I suspect its more like that newer ML-derivatives will. I like ML, too, but I don't think ML -- or at least Standard ML -- is likely to have a big renaissance. I think instead that newer ML derivatives or languages influenced by ML -- languages like Haskell or Rust -- will have their day.

Let's not forget F# and OCaml. Places like Jane Street become very attractive for the sake of working with OCaml and (presumably) people with a high level of skill.

Haskell is very attractive. Good run-time performance, the ability to avoid common errors through higher-order functions, and the ability to abstract complex patterns.

I looked into Rust and Kotlin a month ago for a Summer coding project. They were intriguing but not enough to choose. I wound up going back to Scala, which I'd set aside for a couple of years. (Scala pays some tribute to ML as well.) I may go back to one of them at some point.

SML is old, very old. SML has quite a few implementations and MLton is one. This is a fork of MLton with some special features. There's also MLkit which has region types similar to Rust. I don't know if MLkit was first though, but the idea existed before Rust, so it's possible.

MLkit is older than Rust. Here's a publication from 2006: http://www.elsman.com/pdf/mlkit-4.3.0.pdf

Strangely MLkit doesn't have AMD64 support yet. That's why I use MoscowML for the REPL, besides MLton for final binaries.

I'm a fan of Poly/ML myself. MoscowML is cool, but it seems like the devs either don't have the time or the will to give it any love anymore, and it has been languishing without any updates for years now. It doesn't even build on Cygwin anymore. Meanwhile, Poly/ML is still maintained, and considerably faster (though I couldn't make heads nor tails of its parser code...).

MLkit was a key influence on Rust. It's no secret that Rust has drawn from prior art in regards to regions - Cyclone is another one.

I think everyone missed that you held up ml as an old language:

> Perhaps someone can explain to me why we need new languages, in the face of rich, powerful, languages/tools such as this.

So it's more "why rust, when ml" rather than "why ml, when ocaml"...

Fun fact, first versions of Rust compiler were written in OCaml before Rust was able to bootstrap itself. Admittedly not really relevant to the discussion at hand.

I deleted my comment after reading yours. I missed the key three words: "such as this". You're right, I read what (s)he meant in reverse. (S)he didn't mean "why did people create SML when other stuff exists", (s)he meant "why does other stuff exist when SML exists".

Thanks for the correction!

In the realm of FP compilers, what I like about MLton a lot are its default support for untagged and unboxed integers, reals, arrays, etc. And SML is arguably a nicer language to learn than OCaml.

I'm using MLton for executables and MoscowML for the REPL, but I wish it was easier than that because you have to come up with some tooling to use both in the same project.

This is a real problem with ML. There's competing tooling from SML/NJ and MLton, not to mention interesting, useful projects like MoscowML or PolyML. So finding reusable libraries and such is a real battle, and you can't escape knowing both MLBasis and CM, as well as SML/NJ extensions and differences in the respective Basis libraries. The underlying "problem" is that MLton is too good to ignore, but takes so long for even small programs that it's impractical for day-to-day, let alone interactive development.

That's what's so exciting about new work on MLton like this: A JIT-ing MLton, for instance, could be an amazing thing. It seems like the community has been coalescing around MLton against all odds, and any improvements to it are a big deal.

SML is nicer syntactically than OCaml, but OCaml brings so much to the table that it's hard to ignore (a simpler compilation model, niceties like named arguments, single development target, good community). Also Paulson's ML For the Working Programmer explains SML's warts well, in a historical context. It goes into great detail of some of the limitations of SML's module type safety and some of the nasty things you need to do to make functors generic and reusable, which really turned me off of the language and explained why Caml and OCaml exist. I still love SML but must concede that it's mostly a teaching language. The Little MLer is wonderful.

The Reason team at Facebook (http://facebook.github.io/reason), I myself included, are working on the syntactical aspect & more. Lots of OCaml's pitfalls are low-hanging fruits (imo), and we're actively fixing them.

The module nastiness is supposedly solved with 1ML, but OCamls folks are not standing still. I mean, kacey's effects type system looks like an elegant way to build concurrency features, and I'm glad they're basing OCaml multicore support on that.

What are your thoughts on Ur (with and without /Web)?

There is no such thing as Ur without /Web (and unless things have changed recently the author has no plans for decoupling them anytime soon).

Ur/Web also lacks incremental compilation, rendering it unusable for general purpose real world applications.

Otherwise, as a modern ML that has improved upon existing MLs (Hakell included), it would be great to see it, 1ML, or other modern spin on SML take root in the FP community.

Via https://twitter.com/silentbicycle/status/751574435199672321, I found http://arxiv.org/abs/1601.03116, which talks about how they do the realtime GC stuff.

An interesting bit for me is Section 4's way of handling fragmentation. They have a hard max size on allocations: large objects are linked lists and arrays are managed as trees (32-ary trees, so lookups are often not super deep). There's an obvious price in locality and pointer-chasing, but if consistent pauseless GC (and not throughput) is your goal, never having to compact to make space for a big allocation is neat.

I know ocaml has a lot of proponents in industry (jane street probably the most vocal, but I've seen it many other places, as well). Most of the SML/MLTon references I come across seem to be in an academic and research context. Is anyone out there using it for "real world" application development? What sorts of niches would it serve well?

I once prototyped an acoustic ray-tracer in MLTon, and it was a pleasure to work in.

I have no idea how close I could have gotten to C/C++ -level performance, but I found debugging the prototype to go way faster in MLTon than it likely would have gone in C/C++.

Unrelated, but...

Acoustic ray-tracer? That sounds like fun. Is there anywhere I can learn more?

Here's one place: http://oalib.hlsresearch.com/Rays/

If you want to actually understand it, this book is pretty good IIRC: https://www.amazon.com/Computational-Acoustics-Modern-Signal...

Reply on news.ycombinator.com