Introducing MongoMallard: A fast ORM based on MongoEngine | Hacker News Reader

Introducing MongoMallard: A fast ORM based on MongoEngine

hack.close.io

51 pointsthomas-st13 comments

I used Mongo in Python for a couple of projects recently. I tried out every single library I could find. In the end I settled on using Schematics [1] with a very thin data layer for loading and saving the objects. It worked really well. You get to work with sane Python objects and it's really easy to move between Mongo and Python.

Ultimately, two projects later I'm just not happy with Mongo as a data store. I've found that the restrictions of a relational data model mean that you don't end up paralysed worrying about how to store your data. In Mongo I was forever tossing up between embedding data or splitting it up into a separate collection. And it's a choice that may well bite you later. Also, only ensuring the integrity of your data within the application is not a great idea.

[1] https://github.com/j2labs/schematics

Regarding schematics: We actually developed a object validation library that's very similar to it. It's called cleancat. It comes with some MongoEngine-specific fields to validate references. It's best to look at the unit tests to see how it works: https://github.com/elasticsales/cleancat/blob/master/tests/_...

Regarding collections and embeddeded documents: Yes, a lot of times it makes sense to split things out into separate collections instead of embedding them so you don't run into problems later. One nice thing about MongoEngine and MongoMallard is that they update only fields that have changed. So if you have a ListField(EmbeddedDocumentField()) and change one of the embedded documents, only that document gets sent in the update statement, which speeds things up. And yes, I wish MongoDB had better ways to ensure data integrity.

Schematics was an early fork of MongoEngine - focusing on data modelling and validation of various data formats, dicts, json & bson documents.

Interesting. I know it's a rework of DictShield but I was unaware that came from MongoEngine.

Really pleased to have these improvements and look forward to getting them into MongoEngine 0.9 :)

What a nice layout without a hundred different analytics scripts/widgets/gadgets. Even inline css.

Thanks Dorian, we write our posts in Markdown and use simple static content generation: https://github.com/elasticsales/hack.close.io

Have you been working for too long with the new ReferenceField? Because it was an issue before the 0.8 release that you could worked too. Also what version of flask-mongoengine are you using?

Also since the site doesn't have comments include a link to this page in the end?

The thing that changed in 0.8 is that references are now stored as ObjectIds only instead of storing both the table name and the ObjectId (you can still get the old behavior by using dbref=True). However, you would still have to check whether or not the reference was an ObjectId to tell if it was valid.

We're using our fork of https://github.com/elasticsales/flask-mongoengine/ (which I admit is based on a flask-mongoengine version that is a little bit outdated).

Also, will add a link to the comments.

Almost entirely off topic, but is "ORM" a fitting term for something that maps documents, and not relations, to objects?

No, clearly, its not, since its not object-relational mapping.

And one of the selling points of NoSQL "document store"-style solutions is that, as they do not use the relational model, your data layout can more closely follow object construction and not need the kind of object-(datastore)-mapping represented by ORM, anyway.

I've heard the term ODM used for this purpose [0].

[0] - https://github.com/jmkgreen/morphia

MongoEngine prefers ODM but documents often have relationships.

That comes at some cost in MongoDB and document databases in general. Often it is best to start with the opposite of modelling a relational db - start denormalised and normalise where it makes sense to.

The catch with MongoEngine is know that ReferenceFields are dereferenced in the application layer and design your Documents accordingly.

Reply on news.ycombinator.com