Hacker News Reader

Comment by bvrmn | Hacker News Reader

Message bus proponents never build large systems. The only sane way is to be pretty specific about data flow with clear mental model shared between developers and ops. Message bus hides producer-consumer relations and with multiple endpoints it's very hard to reason about the system as a whole.

View on HN

Some of the most complex systems in the world use a message bus. The entire car and air industry come to mind.

Being able to see a message between systems and have an analyst figure out whats wrong and what it needs to be is an awesome way for a developer to actually code.

Why do you say that? I'm someone who's worked on a few very large message based systems in finance for years (100s of devs working together in multiple countries). I found that messaging and workflow orchestration were the things that helped keep things same.

I've seen people succeed in a big way with message bus architectures and I have seen them fail in a big way.

The master symptom I've seen is that people queue work and that work never gets unqueued. It's amazing how many real-life systems fail with the same symptom.

Perhaps that could be solved with monitoring the number of pending requests and alerting over a certain threshold?

That diagnoses the problem, sets the stage for solving it, but doesn't actually solve it.

Isn't that an "easy" problem to solve, in the sense that you just increase the processing capacity of the consumer?

That's a weird situation, I've never seen that happen personally. A good service bus should avoid this.

For example, https://node-ts.github.io/bus/ subscribes it's queue to all message topics that it handles. If, for some reason, a message arrives that no internal handlers exist for, then the message is simply logged and discarded.

The largest system I ever worked on used a message bus for almost everything, and it worked great.

That said, 100% agreed about sanity. There was a lot of time spent on ensuring that all the developers (and there were a lot of them) had a decent shared understanding of everything, and a very deep understanding of anything they worked on or interacted with directly. Meetings to review the current way data flowed and look for ways to clean it up were a regular occurrence, and ops was deeply involved in everything.

The articles that say or imply, "Don't worry about it, because it's easy for everything to talk or listen to everything else!" strike me as having probably been written by someone whose message bus system hasn't been around for all that long.

I see. My original comment was not clear. I also use queues a lot in my systems. But only as implementation detail and it helps to solve scalability and persistence problems. Message bus proponents like to place common bus as central medium to pass data between components aka fire event and somebody (often multiple handlers) will take and process it, easy-peasy, ha.

Absolutely not true, I’m afraid. I’ve architected or worked with message bus architectures in large-scale medical payments processing systems, as a central broker for a national-level telecommunications company, multiple Fortune 500 consumer-oriented companies with complex sales workflows, and so on. This doesn’t obviate the need to be clear on producer-consumer relations, as you point out, but that’s a governance issue, not a technical architecture problem, and one that is at least as severe with API-only architectures.

I've worked directly with enormous systems that you have likely personally used that were not only highly coherent and well designed, but also message oriented in architecture.

Are you aware that LinkedIn developed and uses Apache Kafka for critical production services?

I also work on a large production service that uses a message bus. There are certainly amateurish uses/implementations of message buses out there but definitely not all of them

Reply on news.ycombinator.com