Hacker News Reader

Comment by lab14 | Hacker News Reader

What do you mean "consciousness for AI systems"?

When you consider the message passing paradigm, I can envision how that simulates neuro-chemical communication between neurons (now here being functions that do things). So there is a design for communication between functions, then there are memory storage and information retrieval parts of the system, short-term RAM, long-term HD/SSD, databases, cache-systems, where relevant information can be and become manipulated. Long and short running processes, acceptance of fail-fast and that communications can fail and that's just part of software system life (I assume also a similar idea in the human brain, don't you find yourself forgetting something you were just thinking about?). There is then the external part of the system, accepting inputs from the outside.

The message passing paradigm is called the "Actor Model" and was invented in the 70s IIRC. It's available in most programming languages and it's not something exclusive to Elixir by any means.

https://en.wikipedia.org/wiki/Actor_model

Correct, it's not strictly unique to this. Even Ruby has it, but Ruby does not have BEAM/OTP.

Smalltalk was a fun language to work with, albeit academically.

Having built a mediumish soft realtime queue processing solution on rails I gotta say it was not great. If we didn’t kinda grow into it I would not have chosen ruby

Thanks for sharing. Yeah, I think that's a problem with bolted on functionality. Did you end up building a new version with a different language? What kind of problems did you encounter that made the solution "not great"?

We eventually managed to build something in sidekiq that did the job but if your jobs are not unrelated and you want to make sure not to starve any specific resource the options in ruby are just way worse than eg Elixir

Are you able to provide more context and details? It’s always interesting to learn about system design and architecture decisions from real world examples.

The issue was processing messages for live pro esports tournaments. Each game had to be processed and the biggest issue was that messages from the queue were not guaranteed to ordered and because of processing issues upstream you could get a large amount of messages for a specific match in sequence. Thus we decided to run just a light weight worker on the original external queue copying to our datastore which is very fast and can be run massively parallel.

Then run a unique job per running match that does the ordering and processing. This worked well enough but you need to now make sure that each worker gets called again after it finished and originally we had a recursion going on but the best place to do it was eventually deprecated in a new sidekick or unique jobs version update. We eventually settled on a supervisor that would monitor and restart workers. Which is just a bad version of elixir in the end

Not exclusive to Elixir, but the BEAM was built from the ground-up around Actor for concurrency. Almost all other popular language ecosystems have it bolted on over time (with varying degrees of success) or relegated to a third party library. It's very natural and ergonomic in Elixir, because it's the paradigm of choice in the underlying vm.

Goroutines probably come closest.

> It's very natural and ergonomic in Elixir, because it's the paradigm of choice in the underlying vm.

Thank you, this is the key factor -- a fundamental design decision that influences the foundation.

Reply on news.ycombinator.com