On Redis, Memcached, Speed, Benchmarks and The Toilet | Hacker News Reader

On Redis, Memcached, Speed, Benchmarks and The Toilet

antirez.com

65 pointsjulien25 comments

memcached's ASCII protocol is known to be slower and less effective than the binary protocol and using TCP for memcached is slower than using UDP. It's not a fair assertion that redis is faster than memcached when you're picking the slowest way to communicate with memcached.

Also, one should not expect to see parity in get/set ratios or you're doing it wrong. Cache get with a good hit ratio/high effectiveness is the point of memcached. While memcached gets effectively plateaued in his low concurrency example, redis get performance degraded as you added more clients/concurrency. In an unoptimized use of memcached, high transaction rate environment, I see sets at 10% of gets.

Redis may very well be faster, but an effective benchmark should be closer to real world use case and shouldn't hold one party back for parity in protocol complexity.

Crad: I compared the same protocols, both TCP, both ASCII protocol. This is a fair comparison for sure with TCP, as with UDP you have many tradeoffs, so this is specifically a comparison of the TCP (reliable) protocol you can use with Redis.

About the binary protocol: I think this is currently not the widely deployed one, but the point here is that if the difference is about the protocol, even Redis could switch to a faster protocol using a binary one. Using almost the same protocol in both servers we are actually measuring the internals.

Anyway your reasoning can be applied the other way around. Many memcached systems are currently using CAS extensively while you can perform just one query with Redis atomic operations in many of the same real world scenarios. So should I double the Redis numbers by two because there is a faster way to do something in a Redis specific way?

About SET/GET ratio, I measured both SET and GET performances in an independent way, so I don't get it.

I get your reasoning about ASCII/TCP and it makes sense... apples to apples.

When asking for a performance benchmark, specifically about systems that are very dependent on communication protocol for performance, I think that it's more valuable to know how the best performing protocol of either side would perform.

As far as atomic vs CAS, I'd be more interested in the "right way" than the same way, so for my own interest, yes.

Re: Ratio, from your graph it appeared they were co-mingled. If each was done on their own then my point is moot.

This assumes the people developing memcached decided to spend a lof of time optimizing their implementation of the ASCII protocol, despite knowing that performance conscious users wouldn't use it anyway; while it may very well be true (I don't remember), it doesn't seem like something that is safe to take for granted.

I'm Redis using it for:

- session management

- message queue (previously using 0mq pub sub)

- memory cache (LRU argument = weak)

That's previously 3 technologies I had to manage, now only having to install 1. That means only having to worry about 1 driver, and only dealing with 1 API.

Less technologies in the stack is a good thing, esp if you get a performance boost as well.

The current bottleneck is of course an easy way to scale, and 2.2 comes with Redis Cluster.

The other problem some have indicated is the LRU implementation, which imo is a non-issue in almost all use cases.

Are you using it as a LRU cache? I'd really like to use Redis for this -- are there any examples out there of how to do it / could you share how you've done it?

set max-memory inside the config, it'll automatically do it for you as long as you are using expire on the keys.

The implementation according to antirez is that they sample 3 random keys, and expire the soonest one, so put your high priority data with a longer expire and you'll be fine.

Thank you ^_^

Really don't buy any of OA's weak arguments for speed over scalability. But I do appreciate people taking shots at assumptions. Like mine that "memcached is the fastest."

Not over, we need systems that are both fast and scalable. But I suspect than in five years since now most people will consider a much more reasonable thing to shard client side, by hand, and in the most uncool ways a system that can deliver 100k requests/second than an automatically sharding system with 10k requests/second. My guess is that currently the queries/watt figure is not taken seriously enough.

With Redis we will shortly start working full time at "Redis cluster" so it's not that we don't care :) Just I think the first goal should be to have a very efficient "single" node, and then going the step forward trying to retain the same speed if possible.

Ooof, antirez, get someone to help you proofread these things because in some places they nearly make no sense.

"Stressing scalability so much put somewhat in shadow what in the 80's was one of the most interesting meters of software coolness [...]"

"Unfortunately I also think that our dude is part of the problem, since running crappy benchmarks is not going to help our pop culture."

"Apple to Apple, with the same mouth"

Sorry I'm trying to improve my English but progresses are really logarithmic at this point...

Well also the fact that I try to write this blog posts in very little time (it took me two hours at max, probably even a mother tongue should alloc more time to end with an high quality blog post) does not help either.

I wouldn't stress about your english skills; I would bet the vast majority of us have no problem following your ideas.

Your English is great. I'm a native speaker, and reading your post, I thought you were too. A slightly sloppy one with a couple odd turns of phrase, but perfectly clear and comprehensible. I understood the quoted parts just fine.

That said, please don't make excuses about how little time you spent on it. It might be true, but it shows lack of respect for your audience. Like code, if it's worth writing at all, it's probably worth writing well.

Keep writing, keep coding!

Thank you nkurz, sorry about the excuses about little time, I did not argumented it enough: the fact is that I really love to share what I'm doing using blog posts, so I tend to work hard to achieve something that's worth sharing, but then I'm so excited about sharing it that I write the post as fast as possible!

I should try to resist some more hours in order to obtain a post that is more consistent and easy to follow... instead I tend to push the "publish" button and even when the post is already live I continue fixing it... Thanks for the comment!

> That said, please don't make excuses about how little time you spent on it. It might be true, but it shows lack of respect for your audience.

antirez's readership probably overlaps significantly with his userbase.

For purely selfish reasons, I wholeheartedly commend and encourage his goal of spending less time writing blog posts. :)

> "Stressing scalability so much put somewhat in shadow what in the 80's was one of the most interesting meters of software coolness [...]"

This parses correctly, just with difficulty. It's not exactly a garden path sentence, but it has several small key words which, if missed, lead to a failed parse.

Not really. And he means 'metrics' not 'meters'. You can understand it, sure, with a bit of effort. But 'correct' it isn't.

Anyway, my point is not to send the guy to study English instead of doing whatever it is he prefers doing. But if you write in public, just get some help proofreading. Despite what the other defensive people on this thread are saying, good, clear communication matters and helps.

Not really. And he means 'metrics' not 'meters'. You can understand it, sure, with a bit of effort. But 'correct' it isn't.

A "meter" is, among other things, "a measure, gauge, or criterion"; and in this way it can take the place of "metric". Idiomatic it certainly is not; but it is correct.

No, it isn't 'non-idiomatic', it's just wrong. You could perhaps find some obsolete or highly technical usage but 'can take the place of metric' is simply inaccurate, in the context and usage. Take a look at

http://www.merriam-webster.com/dictionary/meter

which while not exhaustive is quite indicative of standard usage.

We can both cite dictionaries. My quote came from the OED, which is a far more exhaustive index of English usage than Webster. (Deliberately so: The OED was intended to be descriptive, while Webster believed that dictionaries should instead be prescriptive.)

The modern Webster is not intended to be prescriptive. 'meter' in the sense you are describing is just not part of modern usage. Finding some obscurity or archaism in the OED does not make a lousy sentence 'correct', that's all.

Wrong again (as you've been this whole thread): we use meter in that sense all the time: "gas meter" (both the name of the thing that measures the natural gas usage by a household and the thing that measures the remaining gasoline in a car) uses "meter" in that sense.

Yes, really. The original sentence was this:

"Stressing scalability so much put somewhat in shadow what in the 80's was one of the most interesting meters of software coolness [...]"

This parses perfectly fine with this meaning:

"The stress people put on scalability now has shadowed one of the most interesting measurements of software coolness in the 80's."

I'm not saying that the sentence can't be improved, but that you're wrong when you say that it's nonsensical.

I'm thankful he took the time to learn english and wrote this post in english, because lord knows I have not bothered to learn HIS language, and would not be able to read it if he hadn't.

So, a few turns of phrase, or whatever, who cares? I understood it, and I was able to understand it because HE is not as LAZY as I AM! (and went out and learned english.)

Reply on news.ycombinator.com