Choreographic Programming in Rust | Hacker News Reader

Choreographic Programming in Rust

github.com

52 pointso8vm44 comments

This seems to be a popular idea recently. After reading this paper[1] I ended up building this into my actor based language[2]. It makes a lot of sense in actor based concurrency, I'd be curious to know what rust devs think of this since it's a bit different from how they often do concurrency.

[1] https://simonjf.com/writing/eventactors.pdf

[2] https://march-lang.org/docs/choreography/

Have you looked at Microsoft Orleans by chance? I'm curious what you think of it. The concept is Virtual Actors by Microsoft research.

https://www.microsoft.com/en-us/research/project/orleans-vir...

I had Claude implement a similar one for me in Rust which I'm pausing for now, I'm prototyping a simpler version of Orleans in Go that I eventually want to port over to Rust as my v2 from scratch. The ideas in Virtual Actors reminded me a lot of Erlang, but simple to use in a language like C# I'm still surprised it didn't take off in other languages.

Honestly, if Go supported message passing between processes and over the network out of the box, it would make for a slightly more interesting language, I still love Go though.

This is an interesting question! I'm not an expert in Orleans but I have looked at it alongside koka. The user defined key vs pid distinction is important and using a user defined key like in Orleans has advantages for keeping track of their virtual actors(grains). Their automatic lifecycle management is nice, but you really have to trust the runtime to know when something is really idle. I think their failure handling with auto state restoration is a mistake, recreating an actor with bad state is a recipe for crashes.

What I have that Orleans is missing is compile time protocol checking, typesd and hash checked remote calls, capability management, cluster authentication, and no stop the world GC. Orleans doesn't have choreography per se, they have sagas as a convention.

Yeah, GO routines are meant to solve different problems than share nothing actors.

Nice! I might have to check out yours sometime, I did think about converting a language I built up for .NET with a Python inspired syntax to treat Orleans as a first class citizen as an experiment to see if I could build a sort of DSL for Orleans.

https://github.com/Giancarlos/Culebral

I had to pause for now, been building other things I have wanted made for years with Claude, Culebral was my dive into seeing how 1 million tokens in Claude could accomplish, I think current state took under 1 or 2 million tokens from Claude total.

I know it sounds silly, but the only .NET Pythonesque language I saw available is basically abandonware iirc so I wanted something that was more .NET specific so I could write less code to do the same as I would in C#.

That's pretty neat. I love the things I see people doing like this where they build up some cool functionality behind a nice and straightforward syntax.

Yeah, claude can get pretty far into language projects on 1 million tokens.

Wow, March is cool! Are you in touch with Ashton Wiersdorf? He been developing a choreographic programming DSL for Elixir, called Chorex: https://github.com/utahplt/chorex

Thank you, I really appreciate that!

I'm not in touch with him, but I have looked at Chorex. I was hoping to see him at ElixirConf but I had to change my plans last minute. Have you used Chorex?

Not directly, but we’ve talked about it a few times. He’s in the choreographic programming zulip channel - maybe you could join and tell us more about March? https://multiparty.zulipchat.com/

Thanks! I’ll take a look tomorrow.

*edit it seems like I need an invite.

I would really like the README to be revised in human English. Claudish makes my brain averse to absorbing the information.

that's astrish not claudish

"It's some form of Claudish, I can't read it."

"There are few who can. The language is that of Opus 5, which I will not utter here... In the common tongue it says... err.. actually I don't know what it says."

we need an AI generated movie remix video for this one

That brought a smile to my face. Nice one.

Can't they train these LLMs to produce human-readable English?

And human-readable code, for that matter.

Not well. Xiaomi published a detailed report of their RL process for their recent MiMo v2.6 model (https://news.ycombinator.com/item?id=49792730), which I read through recently. It is an interesting insight into how constructing RL environments work -- for some things, like code generation or code exploitation, there are naturally-perfect RL environments (Github issues + the closing PRs for the former, CVE databases + getting smaller language models to generate exploits triggering them for the latter) -- but for some other things, like code /quality/, Xiaomi at least (and presumably Anthropic et al.) simply do not have any objective criteria.

So they generate a code "rubric" and give a model some number of gold stars for how "well" another language model thinks it satisfies specific enumerated aspects (for code, this is gonna be things like using language features idiomatically, compactness, etc; for writing, this is gonna be things like what you'd get graded on in writing assignments in the fifth grade). Incredibly stupid. I'm shocked it works at /all/. But the lack of any truly objective criteria of "quality" wrt. code/prose explains why it doesn't work all too well, IMO.

I have no idea what this project does, and parsed none of the readme!

It essentially lets you define a communication protocol where things have to happen in a certain order. At compile-time, your ordering is checked against certain rules, and certain classes of deadlocks and hangs can be avoided. The order of actions is then enforced at runtime.

Sounds similar to imperative programming!

... in what regard?

> things have to happen in a certain order. At compile-time, your ordering is checked against certain rules, and certain classes of deadlocks and hangs can be avoided. The order of actions is then enforced at runtime.

This is how imperitive languages work. One line executes before the one below it as a default.

Choreographic languages are often hosted in a functional language like Haskell and OCaml though so the analogy isn’t quite right

I would instead say that choreographic languages enforce global visibility of communication. The global behavior of the system is available syntactically. It doesn’t necessarily or even typically involve mutation, explicit loops, side effects etc at least in the literature.

I appreciate the details! And alethic's explanation in the comments was easy to understand. The key part I missed was that it's a messaging system between software processes. And your insight that it's used in functional programming helps too!

Ah yes, separate software processes is probably the confusion here. Avoiding certain classes of deadlocks and hangs is also something that's not particularly guaranteed by imperative programs. You can throw a type system on it and get some of them (session types I think are quite expressive), but in practice I don't know of any imperative languages with a type system rich enough to do this.

IMO the README is not Claudish, this project is in an area that is more "computer science" than "coding" and there is simply a high bar on whats needed for prior knowledge. I think you could say ELI5 "Explain Like I'm 5" and it probably is possible to do so, but I don't think its correct to say "claudish".

I read it and I understand it, but the style of the writing is anathema to me.

I'm noticing an awful lot of tricolon, i.e., naming things in threes. Ive gotten the impression that LLMs are really fond of that device.

Pangram rates it (up to "This README is arranged by responsibility:") as 100% AI generated. I'm struggling to disagree.

I really wish our culture to explain the applicability of the things we invent would be way way waaaay stronger.

Abstractions should be earned.

Not because executing them is expensive but because understanding them is.

Choreographic programming is well-known within the programming languages literature, which is the audience this README is intended for. The level of detail here is fine, not everything is written with being posted on news.ycombinator.com in mind.

To explain it briefly though, choreographic programming is a method of writing distributed programs that communicate in some message-passing fashion by -- rather than writing separate programs that communicate with send/recv and must be very intentionally designed to be in sync -- you have /one/ unified messaging primitive. In a typical distributed system, you might have something like A.send_to(B) / B.recv_from(A). In a choreographic system, you'd rather have something like A ~> B, where send_to and recv_from are /projected/ from ~> into separate programs during the process of compilation.

You might want this kind of abstraction for various reasons, but the biggest is that many choreographic systems offer a guarantee of deadlock-freedom. This isn't without tradeoff. At the moment, choreographic systems (and session types, another distributed abstraction) are very bad at handling any kind of failure / degraded performance over the network. There is active work within the PL community on resolving these issues, but for now, choreographic programming is primarily useful for prototypes, rather than production systems.

> not everything is written with being posted on news.ycombinator.com in mind.

The creator of this posted it here, though. So maybe if someone is posting something here we can ask that they include sufficient context for the intended readers to understand it.

Thanks for elaborating, I knew already what it is and is actually adequate architecture for some cases. I just complain at how is explained in an exclusive abstract level instead of an approachable dance of balancing definitions with real word examples to give our brains a break and let them breath bits of real world.

If one of the humans involved shows up here, I have a few questions:

How are you doing endpoint projection?

What is your KoC strategy?

What existing research systems (Choral, MultiChor, Chorex, etc.) did you take as inspiration?

What benefits does your system have over ChorRus?

Also: is this actually a choreography? Or are you doing runtime monitoring to ensure deadlock freedom?

How are failure and leadership handled in choreographic programming? Is it up to the runtime to decide? E.g., if I start a new Bob (Bob prime) in a Bob and Alice exchange but the old Bob still exists, what happens? I guess the transport, failure handling, and leadership are all outside of the scope of these libraries? (someone linked a similar looking chorus library).

Curious how this compares to Rust's existing choreographic programming library, ChoRus [1]. Those of you asking for motivation might enjoy the accompanying paper [2], which is really well written. Lindsey Kuper's group has done a ton of great work popularizing choreographic programming as a DSL.

You can get even more power if you integrate choreographic programming directly into the compiler - for an example of that you can see Choral, an extension of Java that I've been contributing to for a few years [3].

[1]: https://github.com/lsd-ucsc/ChoRus [2]: https://arxiv.org/pdf/2311.11472 [3]: https://github.com/choral-lang/choral

Just had a look. The OP library claims to be no_std compatible while ChoRus is not.

ChoRus describes local computation and communication together in one program. Hibana describes the communication protocol; you write each participant's Rust code separately. Hibana then checks their sends and receives against that protocol at runtime.

I hope that eventually choreographic programming is baked into language runtimes to the point that it feels like manually placing certain data on certain parts of RAM/cache. It's one of those ideas that's great, but manual implementations always seem like a mess.

Hibana now has the capability to build QUIC/HTTP3 in the manner of Hibana Choreography. Check it out here: https://github.com/hibanaworks/hibana-quic

multiparty session types are relevant to this: https://dl.acm.org/doi/10.1007/978-3-030-50029-0_8

The problem with Claudish and this README is that the whole thing is vapid text adding up to nothing in particular. When AI takes over a project like this it really has no goal.

> Repeated regions add no iteration field to endpoint types, descriptor rows

So what? Is this a goal or an admission? Is this something you implemented or what was done before?

Yes, but what does any of this mean?

Do you need to do an interpretive dance ?

Reply on news.ycombinator.com