Hacker News Reader

Comment by mahmud | Hacker News Reader

Seriously. Roll your own over Redis.

Redis is faster than BeanStalkd, a tiny message queue server written in C that uses libevent and a simple memcached-like protocol. Redis kicks ass, specially the beta versions.

View on HN

If Redis uses TCP it will bottleneck long before 0MQ does, because 0MQ uses a reliable-multicast protocol for communication over the LAN. As discussed in the linked article, in the comments.

Also, Redis is a centralized server and a single point of failure - 0MQ doesn't force you into that topology.

beanstalkd is not really a message queue, its a job queue. I understand that job vs message can get into semantic nonsense, but in essence a message queue can and will send any given message to n people. A job queue will only send it to the first requester. Redis does kick butt, but a k/v store is not the same as a broacast (or 1toMany, or many to one) meassage store, is not the same as a 'give me the next job' (1toAny) server either.

Redis is a k/v store, not a queue.

Actually Redis is a bit more than a k/v store these days. It provides implementations of a variety of basic data structures, including lists that support queue operations. There's BLPOP[1] and BRPOP for blocking job/message requests and LPUSH and RPUSH for placing a job/message on the queue.

That said, I'd still like to see benchmarks showing Redis beating Beanstalkd. Of the two, I've only used Beanstalkd, but it's a great example of keeping things simple while still providing advanced functionality.

1 http://code.google.com/p/redis/wiki/BlpopCommand

Key == mailbox/slot, and value == message. All you need are concurrency primitives, and broadcasting (in the case of a distributed queue)

People have modeled message queues with far more primitive solutions:

http://www.freeopenbook.com/php-hacks/phphks-CHP-5-SECT-18.h...

A Redis based solution might not be as elaborate as RabbitMQ, but it will be very clean and straightforward, not to mention blindingly fast, and will get the job done.

Well, I'm quite skeptical about the "blindingly fast" part (network overhead?) but agree that in the light of the current mq-options that's indeed one of the more appealing ideas.

That said, I'm not really in the mood to reinvent that particular wheel myself here, but would probably give someone else's shot a try.

How exactly would you do this with Redis? Redis does not push out data to N > 1 listeners/observers/subscribers. I suppose you could use either SYNC and pretend to be a replication slave, MONITOR and watch for RPUSH, or BLPOP. BLPOP hangs/blocks the client though.

Sure, the Redis C codebase is fairly approachable. Perhaps you were suggesting adding new protocol commands?

By what metric is redis faster than beanstalkd?

Does redis has any job queue facilities for making a reliable, low-latency, safe job queue?

http://www.rockstarprogrammer.org/post/2008/oct/04/what-matt...

Yes, it does. See the LPOPRPUSH and BLPOP protocol commands. There's a number of threads on the Redis Google Group about using Redis as a job queue. There's also Resque by a githubber.

http://github.com/defunkt/resque

RPOPLPUSH does about half of what's needed and BLPOP does the other half, but neither does the whole job.

It's not interesting to discuss what you can use to hack a queue together. Lots and lots of really badly thought out queue services exist today built on top of memcached or a relational database or other things that appear to work in normal cases, but can go horribly wrong and start losing data when bad things happen.

Reply on news.ycombinator.com