Clojure Concurrency (2008) [video] | Hacker News Reader

Clojure Concurrency (2008) [video]

youtube.com

76 pointstosh23 comments

I only watched the first few minutes because I have a lunch date soon, but thank you so much for this link. I've been wanting to understand the basics of Clojure and I can already tell Rich's oratory abilities are exceptional. I'll definitely be watching this video in its entirety.

I also found the introduction in the beginning quite interesting to listen to. Good summary of the motivation behind Clojure and its design.

there are aggregations of Rich's talks on clojure. Here is one: https://changelog.com/posts/rich-hickeys-greatest-hits

I recently built a side project that utilized core.async. My thoughts on it are that the use of go-style channels combined with immutable data structures makes for a really nice experience when reading and writing code.

My main problem was with the frustrating debugging experience: Clojure is so concise that breakpoints can be difficult to place, and macros are almost completely off limits. On top of that, figuring out which part of the code would pick up after blocking on a channel was just tiresome. These are the main reasons I decided not to continue with Clojure personally.

> breakpoints can be difficult to place

I can see that for sure, but don't avoid macros. In fact, I think roll your own debugging and logging are their best use cases.

Great post on this here:

http://eli.thegreenplace.net/2017/notes-on-debugging-clojure...

It's probably not what you want to hear, but there's a "I get it" moment I reached at one point, where my intuition just all of a sudden was able to quickly pin point all my errors. There's a lot of these moments in Clojure. There's one for reading the code, one day you wake up and you don't see the parenthesis anymore. There's one for prefix notation. There's one for recursion, where you suddenly figure out how to convert any loops to it. There's one for the documentation, it goes from not making sense to perfectly explaning how each function works concisely and clearly. And there's one for error messages and debugging. There might be more.

So, the community and core team is aware of the learning curve hump, and always trying to improve it, but after you get over the learning hump, the challenges of debuging almost go away, and it becomes equal to any other language. At least it did for me, and it was almost an epiphany.

`(pst)` prints the last stack trace, which can really help with debugging. I only learned this after two years of Clojure.

> On top of that, figuring out which part of the code would pick up after blocking on a channel was just tiresome.

This can be a sign that a construct other than a channel is better suited to the problem at hand. Options include future/deref and promise/deliver.

For debugging Clojure, I use a mix of:

- REPL-driven development [1]

- Platform-specific debuggers [2] (e.g. IntelliJ debugger on JVM)

- Logging infrastructure

[1] https://vimeo.com/223309989

[2] https://www.youtube.com/watch?v=ql77RwhcCK0

Ant Simulation Code: https://github.com/juliangamble/clojure-ants-simulation

Concurrency question:

Is their a good rule of thumb for this? In a web app, how much concurrency should one pursue? If a lot, one user can potentially hog all the machine's cores while computing a view function. If a little/zero, then the server is computing view functions slower than it could if it used more then one core or thread per page view.

for a web apps concurrency is usually abstracted away into the database. concurrency within the web worker is usually boiled down to parallel access to separate blocking resources that don't interfere with each other. it's quite rare to see web worker sharing some state between requests (because as soon as you hit multiple workers you need to care about session stickiness, not to mention the fact that state itself becomes sharded, so not very useful for most purposes) or request having to do enough heavy lifting to require parallelisation.

it's a capacity planning issue which is somewhat related to how you parallelize the work but in the end, roughly `work per request W * number of requests N must be <= worker capacity C * number of workers M` or you will have a growing backlog. i imagine sync vs async resource access affects request timing variance more than it affects the median when the system is saturated but that depends on how resources respond to contention.

so to answer your question - there is no general rule, every system has it's own scaling characteristics, but simpler system is better than complicated one so start with sequential access and measure a lot as you change it.

makes sense. given how much folks talk about concurrency you'd think it would be at top of mind from the beginning of a project.

did the situation change since 2008?

All the stuff discussed in this talk is still basically the same.

A few years ago they added the "core.async" package that provides something like the channel-based, CSP concurrency model that you find in the Go programming language. For many problems, this model is a better fit, but all the original concurrency stuff is there too.

STM is not sold as a headline concurrency feature anymore, apparently people prefer more traditional ways.

I think the main thing is that the use case of modifying multiple references at the same time where it is critical that all changes take place at once is less common in idiomatic Clojure than people might have expected.

Yeah, this is exactly it. Atoms were added as a cheaper alternative to Refs for when you didn't need transactions, just compare-and-swap. Turns out that atoms work well enough for even huge single-JVM systems, and are much simpler.

Since most applications use a database, is simpler to keep transactions only at one place, one source of truth.

for 'business logic', sure, but things like updating/invalidating application caches, or other operations intrinsic to the application (the "non-functional requirements") that can't be isolated to the database.

They have added reducers with parallel folding, core.async and later added transducers and direct support of them to core.async too.

Perhaps not directly related to 'Concurrency' but iirc ClojureScript got released in 2011

https://clojurescript.org

https://www.youtube.com/watch?v=tVooR-dF_Ag (ClojureScript Release)

Not to this, but some things got added to the language:

* core.async * pulsar

Also a lot of the stuff just didn't turn out to be as common as anticipated when idiomatic Clojure is used.

Reply on news.ycombinator.com