leif

My hope is that now that the code is open, more users will start pounding on it and publishing benchmarks. Justin Swanhart recently posted some great benchmark results where he showed us in a less-than-flattering light: http://shardquery.com/2013/05/25/tokudb-vs-percona-xtradb-us.... I haven't studied the workload he ran, but it looked like he may be pointing out some bad query planning behavior and slow cache warmup properties that we should probably address at some point. And that's good for the product and good for the ecosystem, so I am thrilled* that he posted it. I just hope we get 50 more just like it.

Anyway, I'm glad you've got a better understanding. Hope you can find a use case for it!

The statement is really pretty much true. The best B-tree indexes (InnoDB) beat Fractal Tree indexes a little bit on in-memory reads, but we're not done tuning our implementation. On out-of-memory reads though, we usually beat B-trees because we compress so much better that we simply need to read less off disk.

Benchmarking is a tricky business though. Workloads can be varied, and you can probably find some corner cases where InnoDB beats TokuDB. I think we have some outstanding issues where the MySQL optimizer doesn't plan our queries properly, and we're still working on that. I'm a little out of touch with the MySQL side these days though so that could be wrong. But algorithmically, there are no cases where a B-tree index has a significant advantage over a Fractal Tree index.

Generally though, the read performance advantage we see is that if your indexes are Fractal Tree indexes, you can afford to maintain a richer set of indexes than you could have on InnoDB or MongoDB, and these extra indexes make your queries orders of magnitude faster. I think this is the most important (non-obvious) point to understand. I gave a talk about it here: http://www.youtube.com/watch?v=q6BnG74FZMQ

Though you are right, the architecture we tried earlier, when we were doing a proof of concept, would not have helped with compression. Back then we were trying to play nice with mongodb's data format and keep its legacy indexes around. It turned out that for many reasons (concurrency, recovery, and compression, and a few others), it was a lot better to replace all of the storage code, so that's what some of us have been doing the past couple months.

Come talk to us if you're interested. We are stabilizing things and hope to open up evaluations more in a week or two.

you should probably add options for awesome/xmonad, those are fairly popular in some circles.

Don't mean to steal the thunder from RethinkDB, they're doing great work too.

    8. New arrays: SYMTAB, FUNCTAB, and PROCINFO["identifiers"]. SYMTAB allows 
       indirect access to any defined variable or array; it is possible to 
       "walk" the symbol table, if that should be necessary. 

- snapshots (LVM, EBS, etc...)

- cold backups

- mysqldump (with MVCC, this is technically hot)

Also, we're on github: http://github.com/Tokutek and on IRC at #tokutek on irc.freenode.net, so come hang out!

    $ git add .ssh
    $ git commit

Maybe I can be more helpful:

Being a programmer is like being a novelist. You need to be able to construct a plot, but you also need a powerful vocabulary. I'm saying C helps develop your computer vocabulary, and you're saying the programming analog of "but it doesn't tell you how to invent interesting characters!" Totally different parts of a useful experience.

If you want to play with straw men, go elsewhere. All you're doing here is wasting everyone's time, including your own.