The first question I have is why? SQS seems like such a simple thing to keep hosted.
50k messages a second would cost you around $50k a month for AWS sqs, (math could be wrong, didn't double check).
Plus, with sqs, you get what they have. No customizations.
No way they would actually hit that sustained throughput for the entire month.
Even the other justifications about wanting to reference messages after delivery do not to me justify migrating off SQS/Kinesis, especially not at cost of 5 months development effort.
Maybe they have chatty IoT devices. Maybe a ton of sensors monitoring manufacturing plants for multiple different metrics in real-time. Maybe they just have large scale.
Having said that, assuming Pulsar has a similar feature to Kafkas log compaction I can definitely see the appeal!
Also the fact that SQS Fifo doesn’t integrate with this setup is super annoying.
Edit: Log compaction not key compaction