HNHacker News
TopNewBestAskShowJobs

spimmy

355 karma · joined November 19, 2012

CTO/cofounder at honeycomb.io
submissionscomments
spimmy··on MongoDB and RocksDB at Parse
Like I said in the post, we're preparing a series of blog posts for next week that does a deep dive into our workloads, our benchmarks, how to repro, etc.

Spoiler alert: we consistently get around 90% compression and the inserts are ~50x faster. ^_^

spimmy··on MongoDB and RocksDB at Parse
Which data loss issues? You're gonna have to be more specific. ;)

It doesn't fix the election rollback issue, because that's handled way above the storage layer. It does solve a whole slew of storage engine related write issues though. No more "we flush to disk every 100ms and call it good".

spimmy··on MongoDB and RocksDB at Parse
God yeah, the write locks have been a pain in my ass for so long. Fortunately the storage engine API delegates locking to the engine, so we get document level locking in RocksDB or WT.
spimmy··on MongoDB and RocksDB at Parse
Yes and no. (That's my blog post btw.) Mongo is a young database, with some super obvious flaws and growing pains. Some of its bad rep is self-inflicted, when Mongo reps make massively overinflated claims about its reliability, scalability, and performance, and then don't back down when the entire internet laughs at them.

But there are some genuinely amazing things about MongoDB and some places where it really shines. Like flexibility -- we run over half a million different apps and therefore half a million different workloads on Mongo. Schema changes and online indexing are painless. Elections are pretty solid. And the data interface layer really can't be beat.

With the pluggable storage engine API, mongo is really growing up and becoming a real database, much like mysql did many years ago when it graduated from MyISAM to InnoDB. I'm excited about the future.

spimmy··on A Look Inside Our 210TB 2012 Web Corpus
What do you mean by "open"? Can the data be used for startups and other commercial purposes?
spimmy··on AWS Case Study: Parse (YC S11)
Hm, well Sidekiq wasn't out when we started developing. :) Memory has never been a constraint for us with resque though. The only problems we've had with it have been things like improper signal handling, pretty easy to deal with.
spimmy··on AWS Case Study: Parse (YC S11)
We use nginx for serving static files and doing some trivial top-level routing. Haproxy is for load-balancing among families of unicorn app servers.
spimmy··on AWS Case Study: Parse (YC S11)
We use redis because we use resque extensively for the push notifications architecture. Memcache is superior to redis for simple caching in front of data sources.
← PreviousPage 3 of 3