Hacker News Reader

Comment by zerotolerance | Hacker News Reader

There are good things about this article. It seems like a nice intro for people who have never worked on this type of system. And I'm always happy to celebrate someone's win. But I feel like it was missing an overview of what actually makes distributed queues difficult (the distributed part) and why you probably don't need them if you can survive without them.

View on HN

Am I missing something or did the article never address the title?

> How I solved a distributed queue problem after 15 years

Well… how? The post is a nice description of durable queues, but it never explicitly says they’re a solution to a distributed queue problem, nor does it specifically define such a problem.

Is “durable queue” a brand name of a DBOS feature? Because the post doesn’t even say “and here’s how you can use DBOS for durable queues,” nor does it compare it to Kafka or any other “durable queue” solution that’s emerged in the fifteen years since the author used RabbitMQ… (btw, isn’t RMQ a durable queue…?)

Not only is RabbitMQ durable, it has transactions with rollback. The entire set of problems the author describes are down to Reddit using Rabbit like Redis.

RabbitMQ of 15 years ago did not have those features. I have no idea how reddit uses rabbitMQ now, or if they use it at all.

Came to say this as well. I was excited for a deep dive into a difficult problem, and only found an enormously superficial article that didnt even mention the title.

Very disappointing. Like Reddit

edit: I now see that its not a personal blog, but actually just a fluff piece for a company blog. If they had actually gone into detail about the problem, and how the service they offer solves it easily, it would have been fine

Reading it I imagine it's roughly because they started with the problem of "we have to async writes to postgres for scale" and then solved it with "we synchronously write checkpoints with enough performance and guarantees to postgres to solve scale". The middle bit was likely quite hard.

Agreed it's missing that detail. I think it makes sense though that the durable queues shouldn't need strong consistency and transaction isolation, just durability, so the DBs can probably be sharded pretty arbitrarily, maybe operate in lower isolation modes, etc, whereas the DB they need to async writes to probably does need transaction isolation and all that. I'd appreciate if the article would confirm or deny my guess here!

Reply on news.ycombinator.com