Hacker News Reader

Comment by prawks | Hacker News Reader

I think if your understanding of the domain is complete enough that you can map out a relatively wholesome definition of the relationships of data then you're probably right.

I think the advantage the parent is talking about is when you're exploring a domain that you do not have as deep of an understanding in. Schema-less data storage can be helpful in that exploration, as it allows you to dive in and get messy without much upfront consideration. Then afterwards you can step back with what you've learned/seen/experienced and build out your conceptual model of the problem domain.

View on HN

Thanks for spelling that out, I see that it can be a useful exploration tool.

For me though, the act of thinking about a problem in terms of data and relationships helps a lot in exploring and understanding what I'm dealing with. Even more so in an agile setting, where things can and will be changed often - the schema is no exception, it can be changed. No need to cast in stone the first schema you came up with.

But that's just my preferred way of approaching the problem :-)

I share your preference, but I am always curious how others do things. IMO the datastore is the most important piece of a CRUD app, because it is the foundation that everything hangs off of. So from my point of view tracking changes to its structure is extremely important in avoiding major headaches. How do developers manage this without a schema definition? Mongo's popularity has always made me second guess my assumptions about how important I think this is.

See I disagree. As a developer the core of a system for me is not the datastore but the code. I have my domain model in code e.g. User.java and it gets stored transparently and correctly to the database using the ORM every time I save it. The ORM will never compromise the integrity of the database by say trying to store a Date in the wrong format.

So you have to ask yourself what is the schema getting me ? It doesn't help with validation or business rules enforcement since I do that in code anyway. And it doesn't help with integrity since in the case of Java my data is strongly typed so no chance of storing types incorrectly.

I would certainly be happier using a schemaless database with a strongly typed language.

I guess it comes down to where you want your dynamism: in the app code, or in the persistence layer. Using a highly dynamic language with a schemaless database feels very unsafe to me. Similarly, using a strongly typed language with a relational DB is sometimes a pain.

I wonder, when one makes a large change to one's app code in the "strongly typed language + schemaless db" scenario, what happens to the existing data that was persisted before the change, which is now in a format that is incompatible with the new model code?

I'm used to writing migrations that change the schema and update the existing data if needed, all inside a transaction.

I just had an ah ha moment, thank you for that. It looks to me like the trade off is deciding where to put business logic based on what needs to access it. The setup you describe sounds great as long as all access to it is through your ORM. I usually assume access from multiple applications is going to happen at some point.

Thanks for sharing your point of view. I'm still not convinced that it's much easier to change schema with MongoDB. Do you mean before there is any production data that needs to be kept indefinitely? If that's so, then I get it. But if you have some production data and your data model changes, you have two options:

1) Don't migrate any existing data. That means you must forever handle the old form of data when you read it from the db.

2) Actually migrate and transform existing data in the database to the new data model. In which case, it seems easier to do in RDBMS because schema changes are at least properly transactional and you actually now what exact schema you're migrating from.

Additionally, with the relational database, you simply don't have to make as many decisions about how to store your data early on because your data model is less dictated by your usage patterns (because you have joins and other tools at your disposal). In my eyes, that's a big advantage that relational databases have, even for prototyping.

You're putting far too much importance on the DB, when really, it's an implementation detail. It has nothing to do with your app, really, and letting your app be a slave to a schema is an anti-pattern. In the end, all a schema really is a fancier linter.

This is one of the advantages of Mongo. No schema, no longer an issue.

I'm fascinated by this perspective because for any business, it's the data that is treasured more than anything else. For example, as a bank I can afford to lose all the app code but I cannot afford to lose a record of my customers and their balances. Therefore, I would never see data as being inferior to my app. The app can be rebuilt from logic; data not so easily.

In my perspective I don't want my app to even touch the schema. That is not the app's job.

It also means that just because a dev decided that the user object no longer needs the DOB field that that field will be discarded. Even scarier, what precisely happens in those situations varies from implementation to implementation. Someone who is handling the database directly will think many many times before deleting any db column. Even then, he will take a back up. I don't see the same discipline among developers when dealing with the same data, just abstracted via an object programmatically.

We solve this by creating a schema definition for mongo collections, from within the app layer. Now we have easy version control where we want it, not in sqldump files like we used to. That was painful.

Most full stack frameworks have a way to create migrations. When the schema changes over time, you just get a mess with mongo, thats just my observation. Its far easier to manage change with db migrations.

You'd be right except that we also have the ability to create data migrations based on schema changes over time. I'm glad it's not built into MongoDB or else we wouldn't have the (fairly ideal) solution we built to work with it today.

During early prototyping, when I'm still figuring out the problem and how the user thinks, that's when I try not to worry too much about a schema. For me a data model is kinda seductive; of course it's necessary when you know what you're doing, but it draws my attention away from the messy uncontrollable users.

"IMO the datastore is the most important piece of a CRUD app, because it is the foundation that everything hangs off of."

I agree. I think it is similar to when Linux quoted:

"Bad programmers worry about the code. Good programmers worry about data structures and their relationships."

Ok, he is talking about low level systems programming, and databases are a level above that, but I think there is still truth in it.

IMO you can't generalize every problem domain by specifying schema first approach.

I developed a real time analytic solution where the system was suppose to receive unstructured web data. And each client of system can send any type of data & generate unlimited number of reports with unlimited number of fields and all of this in real time. When I say real time, I literally mean all processing was supposed to be done in sub-second. Above all the size of data was 100s of GBs every day. None of RDBMS would have modeled this problem with the efficiency like MongoDB did.

Most of the web related things like analytics, CMS systems or when you need high performance and horizontal scalability, its hard to beat Document DBs like MongoDB.

I found it to really suck with horizontal scalability and high performance. We are paying 3k per month for mongo hosting because its such a pain to manage and it performs so poorly at just 150gb of data that we need 3 shards which I find incredibly ridiculous. I would pick postgres tables with JSON fields over mongo any day of the week.

Reply on news.ycombinator.com