Thanks for your interesting POV on the Redis/Kafka intersection on this. What could be interesting is that while Redis Streams
are certainly totally inspired from Kafka streams, it's just a conceptual thing, so the two things can act in very different ways in practice.
For instance just imagine that I never touched a Kafka system in my life, never used it, don't even know the API, I only read all the documentation they have on the site about the design, to get the higher level picture and combine this with my own ideas about fixing the fact Redis was lacking a "log" data structure.
Pub/Sub + other data structures were not able to provide time series and streaming, but yet Redis streams remain an ADT (Abstract Data Structure), while Kafka is a tool to solve a very specific business case. So the applications have some intersection, but are also very very apart.
For instance you can create infinite small keys having streams, so Redis is good for many IoT usages where you receive data from many small devices. Redis Streams also stress on range queries, so that you can combine the time series with millisecond-range queries.
However, yes, the fact that I added also consumer groups is a way to put this "80% streaming" into a more usable streaming systems more similar to Kafka, for the use cases where:
1) The memory limits.
2) The speed.
3) The consistency guarantees of Redis make sense.
However at the same time, it was a great challenge and pleasure to do what I always try to do, that is to create an API for developers thinking like I'm designing an iPhone, and not some terrible engineering thing which does what it should but is terrible to use (I'm not referring to Kafka that I do not know). So I really hope that what you say "easy to setup and use" will be what developers will feel :-)