HNHacker News
TopNewBestAskShowJobs

mathias_10gen

132 karma · joined March 25, 2010

submissionscomments
mathias_10gen··on It's Time to Drop the "F" Bomb - or "Lies, Damn Lies, and NoSQL."
In the sharded case, at any given moment each object will still live on exactly one replica set, which will have at most one master. You can do operations (such as findAndModify http://bit.ly/ilomQo) that require a "current" version of an object because all writes are always sent to the master for that object. You can also choose to accept a weaker form of consistency for some reads by directing them to slaves for performance. This decision can be made per-operation from most languages.

As for trade-offs: Relative to a relational db, there is no way to guarantee a consistent view of multiple objects because they could live on different servers which disagree about when "now" is. Relative to an eventually consistent system, you are unable to do writes if you can't contact the master or a majority of nodes are down.

mathias_10gen··on Making our MongoDB Code Run Faster
There is a good discussion in the comments about how mongodb uses compound indexes.

http://openmymind.net/2011/5/9/Making-our-MongoDB-Code-Run-F...

mathias_10gen··on Rise of the Mac Git GUIs
You may want to try tig (http://gitready.com/advanced/2009/07/31/tig-the-ncurses-fron...). The main view has the same info (may have to hit g to see the graph), plus it lets you drill down into each commit. All without leaving the comfort of your terminal.
mathias_10gen··on Mongoid_fulltext: full-text n-gram search for your MongoDB models
Based on my quick reading* of the mongoid_search code it looks like it is using a regex query against the keywords array which requires a full scan over the index. In fact the way it is being done is probably far less efficient than just doing a normal regex table scan without the keyword array. If you were to modify the search method to do a normal string query (maybe using $in for OR and $all for AND) you should get much better performance.

*Ruby isn't my native language so I may be missing something.

mathias_10gen··on MongoDB Interactive Geospatial Tutorial
If you haven't seen it already, a quick Google hack for computing the radians to input into MongoDB:

http://www.google.com/search?q=50+km+%2F+radius+of+earth http://www.google.com/search?q=50+miles+%2F+radius+of+earth

mathias_10gen··on MongoDB 1.8 (stable) released
Clients are able to wait for the next group commit by adding fsync:true to getLastError calls (some drivers allow you to add this to WriteConcern). We already have some enhancements to this planned for the 1.9 series.
mathias_10gen··on MongoDB interactive tutorial
Actually the _id is usually created client-side so the timestamp will reflect when the application created the object rather than when it was processed by the DB. The only major exception is when the DB creates the _id for an upsert.
mathias_10gen··on Windows 8 to support Multi-Booting
Take a look at http://forums.virtualbox.org/viewtopic.php?f=28&t=33356 (win7) or http://forums.virtualbox.org/viewtopic.php?f=28&t=9697 (winxp).

Protip: never use VirtualBox's "save machine state and close" feature if you plan to dual boot like this. Its too easy to forget and boot to that OS directly next time...

mathias_10gen··on Windows 8 to support Multi-Booting
I think I've finally found the "holy grail" of dual booting. My laptop now has Windows and Linux on different partitions, and either one can run in a VirtualBox inside of the other. That way, no matter which OS I booted too, I still have access to everything in the other.
mathias_10gen··on Minimongo: A micro-ORM for MongoDB & Python
On a related note, if you like python and MongoDB you might like my MongoMagic library: https://github.com/redbeard0531/MongoMagic. It lets you write queries like this: db.users.find(M.age > 21). This produces standard Mongo query objects so works with any ODM, or none at all. With Minimongo you could do something like Posts.find( 10 < M.comments.votes < 100 ).
mathias_10gen··on MongoDB, Data Durability and Improvements coming in 1.8
Actually that doesn't add any additional durability guarantees in the general case. The only way to use MongoDB durably with a single server is to use the new --dur flag coming in 1.8. I assure you there is no other way.
mathias_10gen··on Why You Should Be Using Virtualisation In Your Development Environment
Actually, MongoDB does leave memory management to the kernel by using MMAP to access data files. Most kernels will allocate a lot of memory to the disk cache, which can make it look like mongo is eating all of your ram.
mathias_10gen··on 12 Months with MongoDB
In that case, I apologize. I have seen people using the same argument to claim that MongoDB is unsuitable for use in ecommerce and assumed you were doing the same.
mathias_10gen··on 12 Months with MongoDB
While I don't support a blanket statements along the line of "every app should be using MongoDB," it is equally invalid to say that "anything that deals with money" has no use for MongoDB. If you look at http://www.mongodb.org/display/DOCS/Production+Deployments you will see a few financial and ecommerce sites. I can tell you that there are even more financial firms not on that list in various stages of production. Even if your entire app can't be written using MongoDB, it is still worth investigating if the time savings of using it for a portion outweigh the costs of using an additional data store. But that is more of a business question than a technical one.
mathias_10gen··on Diaspora Developer Release
MongoHQ and other companies providing commercial hosting of MongoDB only need a commercial license if they modify the server and don't contribute back. We want to encourage hosting providers which is why we don't require buying a special license.
mathias_10gen··on Diaspora Developer Release
We try to be very clear about licensing: http://www.mongodb.org/display/DOCS/Licensing. Please let us know if there is something on that page that needs clarification.

I would like to make it clear that users of MongoDB don't need to opensource their product, only their changes (if any) to the MongoDB server itself. The commercial license is mostly for companies that have strict (if misguided) "no AGPL" policies. Most users do not need it.

mathias_10gen··on 3 reasons to use MongoDB
You actually list one of the advantages of GridFS right there in your post: streaming. If you are serving up a 700MB video, you don't want to have to load the whole thing into memory or push the whole thing to the app server before you can start streaming. Since we break the files into chunks, you can start sending data as soon as the first chunk (256k by default) is loaded, and only need to have a little bit in ram at any given moment. (Although obviously the more you have in ram, the faster you will be able to serve files)
mathias_10gen··on 3 reasons to use MongoDB
Actually it would probably be best to reverse the relation and put user_id's in the event objects. Then you can do queries like this:

    db.events.find({hosted_by: MY_ID})
    db.events.find({invited: MY_ID}) // invited is an array of users
mathias_10gen··on Why Everything Wireless Is 2.4 GHz
They didn't really cover the important question of why modern wifi still uses 2.4GHz rather than the much wider 5GHz spectrum. Anyone living in a crowded apartment building owes it to themselves to get 5GHz routers so avoid interference with their neighbors' wifi. For some reason its getting harder to find 5GHz access points and most online stores don't let you search by frequency band.
mathias_10gen··on Node.js 0.2.0 released
That sounds like a good reason to just leave the data in UTF8 rather than converting to ASCII embedded in UTF16 which is broken by design.
mathias_10gen··on Automating partitioning, sharding and failover with MongoDB
You don't have to set your whole system to read only for the entirety of the backup. There are three ways to avoid it:

1) Use mongodump which queries for every object in the database. It will produce a compacted file, but does not do a point-in-time snapshot.

2) Take a backup from a slave. You can either set it to read-only mode, or just shut it down and copy the files.

3) Run MongoDB on top of LVM. LVM supports almost instant snapshotting because it uses copy on write. Once the snapshot has been made (usually less than a few seconds) you can resume the server for read-write operations.

Note that all three options can be combined. You could make an LVM snapshot of a read slave then use mongodump's --dbpath option to produce compacted backup files.

Please see http://www.mongodb.org/display/DOCS/Backups for more details.

mathias_10gen··on How to design a replacement for C++
C++0x managed to add closures without adding GC. The only issue is that if you want to the closure to outlive the current stack frame you have to capture by value ([=] rather than [&]) or use a shared_ptr (which is then copied by value).
mathias_10gen··on MongoDB Performance & Durability
Since Mikeal decided to take down the comments I've archived what I had in an open tab and put it up on s3: http://bit.ly/diuWH8. If anyone has a newer snapshot please let me know and I'll update.
mathias_10gen··on Multi-core HTTP Server with Node.js
I don't think there is a browser that uses a full thread-per-tab model, so there's really nothing to compare it against. The problem with other browsers is that slow JS or in some cases flash in one tab will slow down other tabs.

Also, the tabs in a browser running many sites are much closer to traditional use of processes. Web application servers running the same codebase on each request seem like a better fit for threads. For one thing, the security model for the two uses of JS are very different.

As for robustness, if doing X will cause a crash and your code does X, then you will just have a bunch of crashing processes rather than just one. How is that better? Wouldn't the real solution be to either stop doing X or fix X so that it doesn't cause a crash?

mathias_10gen··on Multi-core HTTP Server with Node.js
Why are they using multiple processes rather than multiple worker threads? IPC is much costlier than using shared memory, even if it's just passing the initial state.
mathias_10gen··on Zed Shaw on C++
Actually Boost::Python makes python and c++ mix beautifully. I just started playing with it a few days ago and am seriously impressed. I'm planning on using it to write a WSGI gateway to put in front of my bottle.py apps.
mathias_10gen··on Writing Rails code makes me miss writing in Objective-C. Or C. Or Java. Or...
sed?
mathias_10gen··on In defence of SQL
If you (and everyone else who've made the same argument) substitute "Syntactic Sugar" for "Abstraction" and "Turing Complete" for "Relational Model" I think you've just made the claim that every application can or should be written in Brainf*ck or one of the other Turing tarpits. Just because a model is mathematically proven comprehensive, (I'm not 100% sure if relational is but that doesn't matter) doesn't mean it's the most natural way to express/solve your problem. And really, that is the only thing that matters. At least when comparing models and not implementations.
mathias_10gen··on In defence of SQL
Actually the name was very much accidental. Read this post by the guy who coined the term: http://blog.sym-link.com/2009/10/30/nosql_whats_in_a_name.ht...

"it was one of 3 or 4 suggestions that I spouted off in the span of like 45 seconds, without thinking."

mathias_10gen··on MongoDB Performance & Durability
A) Just because something is 'traditionally' done doesn't mean its mandatory. Databases 'traditionally' spoke SQL but I don't see you dinging anyone for breaking that tradition. You've used the Appeal to Tradition fallacy (look it up on wikipedia) many times, and it add nothing to your argument.

B) Durability is an important goal at a system-wide level, but that doesn't mean it needs to be handled at the database layer. In addition to the already mentioned replication and transaction log methods, it can also be handled at the block or fs layer using snapshots, or by admins using backup tools. It can even be handled by having a different Database of Record and using Mongo as a operational store. Mongo as software is agnostic; we provide the tools, but it is up to the user or admin to make the best decisions for their technical and business interests. If another layer of the stack provides sufficient protection against data loss, it is unnecessary to pay performance costs associated with doing it in the DB layer.

← PreviousPage 2 of 3Next →