Writing a distributed, scalable system is really hard, and beyond the API, that is the real value for kafka
Writing a distributed, scalable system is really hard, and beyond the API, that is the real value for kafka
Every single person I know who's done this says it was a fantastic decision, and the "eventually I'll have to migrate to X" never came.
It's a weekend project to demonstrate the concept, maybe a few weeks to really flesh it out and iron out quirks. I imagine if you're willing to use sqlite as a backend for persistence, it gets a bit easier.
You have employees - you try to get and retain the best talent you can. However, every human has strengths and weaknesses, and these may not all be fully visible to you.
Rolling your own vs buying off the shelf is a gamble on future outages.
Will a third-party support and fix the issue, or have a strong community that can help you work through the issue?
If your best engineer builds something that works for long enough to become entrenched, but then carks it, will your best engineering talent be able to resolve the issue? If your rockstar quits, does the team have to pick through the halls of Cthulhu? Does your organisational ignorance of kernel networking suddenly become painfully apparent?
Remember, you need to be twice as clever to debug your code than to write it...