What tools, articles, and books have you found helpful in building monitoring infrastructure for your products?
That said, I think what's more important is to log everything ( use Kafka ), even if you don't know in advance how to make use of them. You may need them later. And you need to break this down into 3 different problems. Data exploration, data visualization, and alerts. And you probably shouldn't or won't be able to find one product that will help you address all those needs - or maybe will be too expensive. You should instead use multiple and all should consume from your events feed/stream. If you want to do it yourself, and you don't have the hardware resources to handle lots of data points, then use two differ sets of services. One that will retain say upto 24 hours worth of data so that you will get predictable, consistently fast query, processing and visualization time and another that may hold data for a longer period. Chances are you will want to understand issues and that happened recently so optimize for that. Good luck :)
We are currently building a product to do that ! We have a prototype and we'll be happy if you'd want to be a pilot/beta customer. Shoot me an email: contact@cloudhawk.io