Understanding the limitations of pubsub systems [pdf]

A paper published by Databricks primarily focuses on why "Kafka sucks" and what if we make the source of truth a database and watch abstractions, then cases we miss with Kafka on resiliency and consistency can be solved.

It argues that modern pub-sub systems:

fail to achieve their goal of truly decoupling writes and readers

violate the end-to-end principle

expose an ad-hoc storage abstraction with a bespoke API and limited power

relevant thread on X -https://x.com/BdKozlovski/status/1991892238335381524