Hacker News Reader

> this is less important and we can spend less time developing fancy ETL processes to make everything fit into one, conformant data model

And that's precisely why modern data processes are inferior to 20 years ago. People reinvent the wheel over and over and spend massive budgets on unnecessary tech stacks that would be alleviated if the time was simply taken to model the data.

A clean data model is about a whole lot more than simply storage space.

View on HN

Agree if people would model everything to a “Grand Unified Data Model” of everything, it would be a lot more efficient... unfortunately that is very hard and puts a massive bureaucracy around data governance and management. It slows things down. I guess a more modern approach is to relax those constraints a bit and realise that some data can be expressed in different ways, and that duplication isn’t too much of a concern because storage is cheap. That said, it’s not an excuse for reinventing the stack over an over. I think the thirst to reinvent has largely been driven by the shift from expensive proprietary solutions like Teradata and Oracle to open source ones. That’s a positive shift.

And this is why big enterprises are moving out of data warehouse model for processing all the data and prefer a data lake concept. It is better to centralize some aspects of the data, but definitely not all (like common relational model for all sources, as it is in data warehouse). The data warehouse model quickly becomes super expensive to maintain and evolve.

Reply on news.ycombinator.com