Hacker News Reader

Comment by jadbox | Hacker News Reader

Since this has been around for awhile, why hasn't Redis adopted this strategy into core?

From Antirez's (Redis Maintainer) blog:

> Another thing to note is that Redis is not Memcached, but, like memcached, is an in-memory system. To make multithreaded an in-memory system like memcached, with a very simple data model, makes a lot of sense. A multi-threaded on-disk store is mandatory. A multi-threaded complex in-memory system is in the middle where things become ugly: Redis clients are not isolated, and data structures are complex. A thread doing LPUSH need to serve other threads doing LPOP. There is less to gain, and a lot of complexity to add.

http://antirez.com/news/126

This should be top comment. I came here to chat about potential downsides introduced by complexity of having multiple threads accessing the internal data structure. Until someone runs this in production where they actually use the performance it delivers over and above vanilla redis, I'll probably hold off. I'd like to know it's stable under very high load with contention.

No offense to the creator(s) and I have a ton of gratitude and respect for them pushing the boundaries. But I also know Antirez is a smart dude and Redis has delivered insane performance thus far with few issues.

No offense taken. KeyDB has different goals than Redis so you’ll see us try things Redis might not. I’m willing to make the code more complex if it makes the user’s life easier in some way.

That KeyDB exists and others like it is great! I likely won’t be working on projects that will outstrip the performance of Redis but if I did I’d be looking at the work of people much, much smarter than I for alternatives and from what I’ve seen KeyDB looks really compelling. It’s definitely a project I’ll be following.

Is there a milestone the multi-threaded implementation could achieve that would make it a candidate to merge into core Redis?

I mean the first requirement would be a reason for needing multithreading in the first place. If you can then demonstrate that this implementation solves the initial problem, while not adversely affecting existing performance and stability, then it would probably be a candidate to merge into upstream.

It's a decision choice to be single-threaded. Using threads requires locking or sophisticated concurrent data structures which can sometimes outweigh benefits in both code correctness/maintainability and performance.

Redis is actually exploring multithreading for IO. Here is antirez/Salvatore talking about the performance results in the write path: https://twitter.com/antirez/status/1110973404226772995

https://news.ycombinator.com/item?id=19368955

Previous discussion about keydb, commented by antirez.

Redis unstable introduced threaded IO, but it's still not as fast in my benchmarks.

Reply on news.ycombinator.com