Hacker News Reader

Comment by JPLeRouzic | Hacker News Reader

> "The industry, by and large, puts a crazy amount of effort into avoiding SQL by writing ORMs, translation layers and similar tech "

In my experience (I am retired from a telecom operator) using a database made applications quite inflexible yet real life implies constant changes in their data model and as a result when there was a "database migration" there was always some information lost.

On contrary Pick databases (MultiValue databases) were very useful.

https://en.wikipedia.org/wiki/Pick_operating_system

View on HN

I have done, quite literally, thousands of database migrations in a telco context and we never - not one time - lost information.

I don’t think there is any particular problem, or solution. Applications get old, assumptions change, the world moves in.

It’s true that our database design became inflexible over time and needed to be hoisted. But so too did our authentication layer (hand built - JaaS - JWT), the UI (AWT - Swing - Web 1.0 - Web 2.0), the deployment model (on prem - cloud - docker), even the API scheme (RMI - soap - rest - graphql).

Any application that lives long enough is going to go through this sort of change. The only constant in all of the above was SQL.

Beware the old man in a young man's game. You will be tempted to underestimate him. As many unremembered before you have done.

I'm of the opposite opinion. Database migrations force you to think through the changes in your data more explicitly. You catch logic oversights, boundary conditions and implicit transformations much more thoroughly and with fewer errors later on.

I've seen a lot of different enterprise systems. In my experience it's not the db that's causes this inflexibility issue, but designing the architecture of systems data-first (using Data Dictionaries / ER diagrams etc.) rather than use-case-first that leads to this sort of problem.

When a data schema is the foundation of a system, it's often the most difficult thing to change.

I don’t know how that works. Your data schema is the foundation of almost any system, regardless of how you generate it.

I think this is a common mistake. People think their code is what’s important. But it’s the data and how it gets transformed that is the core of most applications.

Data-based architecture means that you have all data about an entity in a single place. Even though if you analyze the domain and all use cases you find that there are slices of that data that never interact together.

Example is storing customers address and purchase history in same place, only because they "belong to customer" in a data sense. This is very bad because it leads to entangled architecture where everything depends on everything and you cannot extract truly independent modules. The project implemented like this simply cannot scale beyond a certain size.

So no, data is not important but how it is used.

Ok. Perhaps I’m not familiar with this concept of “data-based architecture”. The GP talked about designing around features, which I think is also bad. But maybe they meant more like a DDD approach?

So - I certainly agree 100% with what you say about keeping thing independent. Maybe I’m just unfamiliar with the terms.

A more specific example is the idea of sitting down and listing all the attributes that define a customer profile (name, addresses, date of birth, etc) and then designing the CRUD around those things. Contrast this to starting with what are the interactions that a customer actually needs to worry about (send me this product to the address in my PayPal* account)

*or whatever external system.

If you had to make a guess at how many different pieces of information you have about you in databases for services worldwide (and how many of those are inaccurate, or unnecessary to provide the service), that number would be fairly ridiculous for most people (OOM thousands or tens of thousands).

Taking the address idea - no e-commerce site really needs to store this as a profile item. It needs to be in my browser so I can autocomplete for any shipping form.

The second point I’m making is the same as your last. “How it gets transformed” is often more important to start with than how it is at rest.

(I acknowledge in advance this is a weak argument btw).

I actually agree with all of this.

Ultimately where I’m coming from is that the flow of data through a system and how it is transformed is actually what’s important. For me that’s taking a “data first” approach, to use your term, but I took the term to mean something different to what you meant.

You are totally right about things like customer info. If you have an email component then that’s where email addresses live. If you have a fulfilment component then that’s where physical addresses live. 100% agree that they don’t belong together, at least not by default.

In fact, I tend to think of entities like customers as just a uuid. The bits and pieces we attach to that uuid depend on the system. But many domain objects should be treated in a very abstract way.

I’m not sure this is widely accepted, but it’s where I got to.

For this to be true (which it is in some contexts), the important question is what the boundary of the "system" is.

I agree.

On reflection, I'd also add that I do design systems "data first" however to me this doesn't mean "database first". It's important to think about how data flows through the system, how it gets transformed, and what parts of the system "own" which data.

All of this requires understanding of the boundaries of the various systems that make up the whole, which I think is exactly your point.

Reply on news.ycombinator.com