Hacker News Reader

> I'm straining to see a use case. "Serverless" NoSQL? If you've got something small enough that you want an embedded datastore, why use a NoSQL variant?

Serverless NoSQL embedded datastore aren't really that unusual. BerkeleyDB, Tokyo Cabinet, LevelDB are all commonly used serverless NoSQL.

I think what's happening here is that NoSQL is just an anti-category.

When I think of NoSQL, I think of something like MongoDB or Cassandra which are designed to handle tons of data under heavy load. When I think of BerkeleyDB and friends, I think local persistence. It's a true statement that these too are NoSQL stores, but, at least for me, they're not of the variety that first comes to mind.

The problem isn't the author's lust for tinkering and improving his dev chops. The problem is with HN, how it's designed to drive fads and constantly disturb our focus, and immature developers who change tech stacks as if it was underwear.

The problem? There's no problem here. I find the OP very interesting.

Whether or not it's useful to myself or this community, it's something which someone very intelligent spent a significant amount of time to build. I'd argue that its usefulness is unrelated to how interesting it is. It genuinely makes me want to know more about the author(s) and what their motivation was.

"serverless" databases are usually called storage engines: LevelDB, BerkeleyDB, BitCask, InnoDB. There is a (good) trend towards modular software, separating the storage engine from the server is a natural consequence.

But yeah, requiring a new obscure language pretty much obliterates any reason you'd want to use this. Lua is fast, popular, made for this exact use case and already used by Redis.

I probably came off too harsh. I'm quite familiar with the idea of embedded datastores, SQL and otherwise, and I didn't mean to attack the concept. I was more trying to illustrate (albeit weakly) that the marketing seems a bit off.

Like I said elsewhere, to me NoSQL tends to mean "big, distributed storage thing that isn't RDBMS, usually with an emphasis on horizontal scalability." Others have said correctly that some also take it first to mean schemaless and relax the "big" requirement. However I'm still confused on what value a general-purpose "schemaless" datastore provides in an embedded context. I write schemaless in quotes there because there's still an impedance mismatch between the format in which your application works with and the format in which the datastore works.

All of these rely on some general-purpose abstraction. But your application, if designed properly, probably makes use of a number of varying data structures. What are you gaining by pushing all of these through some abstraction? Assuming that all you're after is simple persistence, I have to ask - is writing data to disk really that hard?

That said, I've used local persistence libraries plenty, usually as a persistent local cache, and/or as a tool to enforce a schema which allows for data migration between constantly updating versions of software.

When I was at Singly working on the locker project, this was actually the absolute perfect solution to how we wanted to store data (it just wasn't available then.) We wanted to store all the raw JSON that was coming down the pipe from various sources (Facebook, Twitter, etc.) and wanted it all to be queryable. Also, since each user's data was completely isolated, having a tiny embeddable solution made a lot more sense vs. firing up a MongoDB or something similar for every user.

There are two different camps within NoSQL; some people are webscale and some people want NoSchema. This looks like it's aimed at the latter category.

I'd have to agree with you here... I mean, there are lots of interfaces for reading/writing json, and simply outputting a JSON as a UTF-8 .json.gz file seems more appropriate... especially if your data can fit into memory.

I would be more inclined to simply have my object structure in memory, using a more convenient high-level language, and do a load/dump from .json.gz files as needed. It's fast enough, and worst case scenario, I can backup/extract and read/write in any number of programming languages.

If you need multiple records, that may be easier, having an index, and using line delimited flat json structures can work as well... (breaking on \n)

This really seems like an also ran, where you could have abstracted out an SQLite db with a single table of (key VARCHAR(100) PRIMARY KEY, value VARCHAR(4000)) or something similar.

Following the thread of building upon sqlite, Yelp (along with the fabulous mrjob library) also open-sourced sqlite3dbm (http://pythonhosted.org/sqlite3dbm/), which is a document store on top of sqlite. Python only (meant to give random-access data to your EMR jobs running with mrjob), but the same approach could be applied to any other language.