Very impressive, especially how it doesn't seem to matter much on the read/write ratio. I've only used redis to cache, do folks really use it as a DB?
Some of it was done with raw Redis commands, and for some we wrote a data-store engine with automatic indexing and optional mapping to models - https://github.com/EverythingMe/meduza
We also used Redis for geo tagging, queuing, caching, and other stuff I forgot probably. It is very flexible, but requires some effort when not used as just a cache.
Right now I'm developing a module for Redis 4.0 that can do full-text and general purpose secondary indexing in Redis. https://github.com/RedisLabsModules/RediSearch
Redis is great because of its multiple data structures. Depending on their "kind", these JSON objects are either `APPEND`ed onto Redis Strings (e.g. for time&sales or order history) or `HSET` (e.g. opening/closing trade) or ZSET (e.g. open order book).
Sometimes an object transitions from a SortedSet to a String. We used to handle this with `MULTI` but now we use custom modules to do this with much better performance (e.g. one command to `ZREM`, `APPEND`, `PUBLISH`).
We run these Redis/feed-processor pairs in containers pinned to cores and sharing NUMA nodes using kernel bypass technology (OpenOnload) so they talk over shared-memory queues. This setup can sustain very high throughput (>100k of these multi-ops per second) with low, consistent latency. [If you search HN, you'll see that I've approached 1M insert ops/sec using this kind of setup.]
We have a hybrid between this high-performance ingestion and long-term storage. To reduce memory pressure (and since we don't have 20 TB of memory), we harvest these Redis Strings into object storage (both NAS and S3 endpoints) with Postgres storing the metadata to facilitate querying this.
We also do mundane things like auto-complete, ticker database, caching, etc.
I love this tech! It's extremely easy to hack Redis itself and now with modules you don't even need to do that anymore.
write coalescing - caching individual writes for later bulk inserts into a db to help insert rates.
Simple counts and counters which can be incremented and then purged/reset on interval.
Simple Pub/sub for medium throughout systems.
Many more applications than simple key/value.
In general, an extra layer on top of Redis is a must to make it even simple searchable database. Especially indices can't be ad-hoc implemented with separate Redis commands. Only transactions or Lua snippets that update both data and related indices atomically can avoid data corruption. For my own use, I wrote: https://github.com/ilkkao/rigidDB It supports indices and strict schema for the data.
I think somebody has described Redis at some point as a database-SDK. Makes sense to me.