Hacker News Reader

leifon MongoDB is to NoSQL like MySQL to SQL - in the most harmful way

Marketing. No technical reason.

leifon MongoDB is to NoSQL like MySQL to SQL - in the most harmful way

Let's see if I can make your day with a shameless plug.

First of all, using the kernel's paging algorithm for a database is awful. It has way less information about your access patterns than the database, so it's going to make a lot of bad decisions, and a bunch of MongoDB's problems with fragmentation and performance cliffs come from using mmap. It's quick and dirty.

Here's the alternative, and it supports the exact same document storage as MongoDB, the same administration interface, actually the same protocol everywhere, hot backup, and however many nodes you want: TokuMX[1]. It does all those things because it's actually mostly MongoDB, just with the storage engine replaced with an engine that works faster, compresses your data, and has more mature things like document-level locking for writers and better multi-document isolation semantics on a single node. Give it a whirl and let us know if you like it.

[1]: http://www.tokutek.com/products/tokumx-for-mongodb

leifon The On-Line Encyclopedia of Integer Sequences

This warms my heart.

leifon Unlocked phones are the way to go

If you read the article, it says they spread out the cost of the phones over 24 months at 0% interest using Apple's financing.

leifon The genius and folly of MongoDB

I've seen a lot of the rest of their code, and most if it is getting better over time, as they grow they're forced to adopt better habits in order to scale their engineering team. I think you're misunderstanding the type of programmers they are. They didn't use mmap because they are sloppy everywhere, they used mmap because their critical innovation was not in storage. What they really thought was valuable, what they wanted to work on, was the query language and cluster management tools, so they did the simplest thing for storage and moved on (personally I don't understand why they didn't just use BDB, maybe they were afraid of transactions, but I suppose everyone has a little NIH syndrome in their database). Now they're a bit locked in to that code, because after bolting on journaling (that architecture is a brilliant but incredibly dirty hack), the code is a mess and I'm sure nobody wants to touch it. In fact most of the other subsystems have been getting cleaner rewrites, except for the storage layer. I think the only way out is a complete replacement, which is what we did so I feel pretty good about that. So I don't know if I'll convince you, but I've read a lot of their code (especially in the last few weeks, I've been backporting things from 2.4), and that's the feeling I get about their history and vision. Hope it gives you some insight.

leifon The genius and folly of MongoDB

RethinkDB is not yet "ready for production use" but reportedly will be soon.

If you're currently on MongoDB but need more performance, concurrency, or compression, please try TokuMX: http://www.tokutek.com/products/tokumx-for-mongodb It's a drop-in replacement server that uses a better storage engine but speaks the same protocol and query language.

leifon The genius and folly of MongoDB

I'm an engineer at Tokutek

I'm confused by your comment. The beginning acknowledges the fact that MongoDB has a weak storage engine, but your conclusion is that, even with a strong storage engine like ours, there is still a problem. What other problems do you see? Are they something we could work on?

leifon Why I moved to Miami

Paris only has good french and moroccan food, san francisco actually doesn't have nearly as good food, so just stop.

leifon Why I moved to Miami

LA has a lot of good food, SF only has good burritos.

leifon Why I moved to Miami

It's a bit of a monoculture, but one that spans more cuisines than anywhere else. There are definitely certain aesthetics that are prized over others that are arguably worthwhile, but it's cross-cutting. You can get more diversity in, say, fried chicken, in PA, but you can't get the same diversity across cuisines, combined with the same depth and quality of execution, anywhere else.

leifon Why I moved to Miami

No, really, you just missed the good pizza here. The offer stands, come back here, email me, and I will treat you to good pizza.

leifon Why I moved to Miami

Come back here and I will show you pizza. Clearly you missed it. Or maybe the donald was your guide: http://www.thedailyshow.com/watch/wed-june-1-2011/me-lover-s...

leifon Why I moved to Miami

No, I've been around the block. There is a legitimate and drastic difference between everywhere else (in the U.S. and Europe, at least) and this city. The only other place in the U.S. that competes is L.A., and S.F. but only for burritos (why doesn't NYC have burritos?).

leifon Why I moved to Miami

It often happens that someone outside new york talks about the amaaaazing food in their city. I used to laugh, or correct them. Nowadays I just feel pity.

Nothing compares to the food of new york city. Absolutely nothing. Don't even fucking try.

"I live within walking distance of two zagat top 10 rated restaurants." There are sixty restaurants within walking distance of my apartment and they're all better than anything in miami. Fuck, 5 of them are run by Top Chef winners. And there are 10 more that are better than those because the food network doesn't even get the top talent.

leifon Hostnames

Characters from Firefly, for my home network. All short and memorable.

leifon The Workman Keyboard Layout

It's reasonable to be a bilingual typist. I use dvorak on my personal and work computers, but qwerty for other people's computers and iphone/ipad. It's really not too bad.

leifon The Workman Keyboard Layout

I learned dvorak by printing out the layout and glancing over at the paper while typing back in high school. It took me about a month to get comfortable and I'm now basically a touch-typing bilinguist (though I'm slower at qwerty, it's more mental effort to do the conversion on the fly since my muscle memory is dvorak).

leifon MySQL is to SQL like MongoDB to NoSQL

I may be biased because I work on it, but I think TokuMX solves or will soon solve all of the "big data scaling" problems that exist in MongoDB.

There is definitely still room for polyglot strategies: Zookeeper is for tiny but crucially consistent data, Riak is for fancy distributed systems availability guarantees when you can afford a simple data model, and I believe Redis has value as a sophisticated programming model for things you can fit in RAM (but I actually have no Redis experience personally).

But in the past few months, I've grown to be really impressed with the document model, and the aggregation framework is getting more and more powerful with each release. I think that's the important part of MongoDB (sharding's a little messy and Riak seems to have their heads on straighter with that), and TokuMX takes that and sands off the rough edges you see with big data sets and concurrency, and I think that's going to end up dominating MongoDB and being a really compelling point in the NoSQL space.

leifon Datomic: Can Simple be also Fast?

Sounds like mongodb's batched write semantics.

leifon Sophia – An embeddable key-value database

How about http://github.com/Tokutek/ft-index? It's embeddable (BDB-like API), has compression built in, and is a similar data structure to this but with more mature features like transactions.

leifon Don't use Hadoop when your data isn't that big

More than 5TB doesn't mean you need Hadoop either, it means you need a better storage system like tokudb/tokumx or leveldb. These technologies can index the data so that you can run selective queries instead of reading all the data for every query like Hadoop would, and they can compress it so that you can keep everything on one disk a while longer.

leifon Happy Petahash Day

Ok, now that we've built a massive compute cluster of GPUs, can we please make it work on something actually useful? Let's fold some proteins or something!

leifon The Trie: A Neglected Data Structure

Mispronouncing variables is great fun. We have lots of "txn_XXX" which we pronounce "texan_XXX", and "le_XXX" (short for "leafentry XXX", as in "le_key"), which always make us sound like we're mocking French people.

leifon The Trie: A Neglected Data Structure

The reason is because the name comes from the word "reTRIEval," which was a pretty dumb move in the first place.

leifon The Trie: A Neglected Data Structure

It's a similar strategy to having one bloom filter with many hash functions, but the math is a little different. Nearly the same effect though.

leifon The Trie: A Neglected Data Structure

Tries (actually pronounced "trees" or "Patricia trees") are very memory efficient compared to most alternatives, but really only shine in membership query workloads. If you actually need to return the stored element, you have to rebuild it for every query, which is probably too expensive if you have lots of queries, it would be better to store the whole object continuously somewhere in the data structure so you could return a reference to it.

Even then, tries only win if the distribution is suitable to give you the memory efficiency you are hoping for. If you don't have many common prefixes, you're better off with just a hashtable.

leifon Your name is way too long for your ID

Thanks, now I need to spend the rest of the day looking for a reason to say "eigenvectors, baby."

leifon Scaling MongoDB at Mailbox

1. > it would be a PITA to write an asynchronous utility in C++

2. Doubt it, and there's no reason for me to write one for them.

leifon Scaling MongoDB at Mailbox

That's not true, I've had no trouble writing very fast threaded C++ programs with the mongo client.

leifon Cocktails for programmers

With initial recipes like these, you'd quickly reach very low local maxima.