> There are various event sources we consume data from, and they are produced in different platforms and storage systems, such as Hadoop, Vertica, Manhattan distributed databases, Kafka, Twitter Eventbus, GCS, BigQuery, and PubSub.

I'm surprised that they mentioned Twitter EventBus, I thought that they were migrating away from that to Apache Kafka entirely. [1] Mind you, they've got a lot of tech going on, so not surprising if it's still present in legacy systems.

Fun fact, if you dig into the architecture of Twitter EventBus, it will seem awfully familiar to the architecture of Apache Pulsar (storage separated from brokers, storage based on Bookkeeper), and that's no coincidence, as Sijie Guo, CEO of StreamNative developed EventBus at Twitter (also a main dev of BookKeeper). StreamNative is to Pulsar as Confluent is to Kafka.

And the reasons that Twitter moved from EventBus to Kafka also apply to Pulsar, which is worth keeping in mind the next time an HN commenter proclaims "Le roi (Kafka) est mort, vive le roi! (Pulsar)".

[1]: https://blog.twitter.com/engineering/en_us/topics/insights/2...