Ask HN: What are the best key-value self-hosted storage engines? | Hacker News Reader

Ask HN: What are the best key-value self-hosted storage engines?

It could be a full db or a file-based binary.

Interested in being: - simple - thread safe - fast - supporting lots of records - querying features is a plus

Tossing another vote in for Redis. Generally, you have to give me a reason why Redis is not the answer for a KV store situation. It can run in a cluster mode or individually, and it can choose to run only in memory and not sync to disk if you don't need persistence and you want it fast.

Some folks claim 200k inserts a second, but we topped out at around 20k inserts a second and had to start digging into more advanced configurations and whatnot. Eventually we sharded and sent some requests to one pool of redis and other requests to a different pool based on consistent hashing that we set up. But even then, I don't think most shops need to worry about that scale.

Is Redis fully ACID? We always used it as a caching engine.

As I recall it’s a bitch making redis highly available in production. But maybe easier now if you find a good K8s operator for it or just use elasticache.

LevelDB: https://github.com/google/leveldb

In Java I like

https://mapdb.org/

It is more feature rich than you want but in Python I'd probably just use sqlite3 since it is in the standard library.

Check out Skytable - https://skytable.io/

Pros: Extremely fast. https://github.com/ohsayan/sky-benches

Cons: Still early in development, not a ton of features.

Redis shines at latency and I don’t see any benchmarks. Ofcourse Skytable throughput will be higher cos it’s multithreaded. I don’t think this is a fair benchmark

Since Redis and memcached are historically the defacto key-value databases I'm curious what about them makes them not as obvious a choice for you.

I'm not saying everyone should only use those two I'm just guessing there's more to the criteria you have in mind than you wrote here.

https://redis.io/

http://rocksdb.org/

Mnesia from OTP (Erlang)

FoundationDB

Reply on news.ycombinator.com