Hacker News Reader

Comment by dustingetz | Hacker News Reader

RDBMS = data silo

You make it sound as if this were a bad thing. RDBMS work well for many use cases. There are plenty of tools around to work with them. Good open source implementations exist.

I played with Denodo (data virtualization software) a couple years ago and thought it was pretty legit.

In theory, it could be used to provide that industrial strength abstraction layer between your Tableau/Looker/etc. and your bajillion weird and not-so-weird (RDBMS) data sources.

That would seem to make sense to me from the point of view of -- I would want my data visualization/analytics-type company to be able to concentrate on data visualization/analytics, not building some insane and never-ending data abstraction layer.

The part that surprised me was that Denodo could allegedly do a lot of smart data caching, thus speeding things up (esp hadoop-oriented data sources) and keeping costs down.

I'm guessing the other data virtualization providers can do similar.

I have had to work with the Denodo for the past 1+ year, a total nightmare. Data virtualization is a "good in theory" concept but "doesn't work in practice" reality. Going back to the original sources for each query doesn't work, it will always be slower than using a proper analytics data warehouse. Caching doesn't help because at that point you can just do ETL. Also Denodo itself is full of weird behaviors and bugs, my team collectively decided it's worth the most hate of all the "enterprise" tools we use. One thing Denodo is good for is as an "access layer", but then maybe PrestoDB would be worth a shot or maybe even just a sqlalchemy and python.

Consider the case when you are a big company and have hundreds of them

The problem space (Business) is not complex, its incredibly simple. Unfortunately the design patterns we use and the tooling is flawed. It has been this way for at least 25 years and the RDBMS, languages, design patterns, and architectures ( 2-tier/ 3-tier) are the cause.

They make it simple to get started and even without knowing what you are doing you can easily churn out something that works if it is simple, doesn't change often, doesn't need to scale and deals with small amounts of data.

"It represents a quagmire which starts well, gets more complicated as time passes, and before long entraps its users in a commitment that has no clear demarcation point, no clear win conditions, and no clear exit strategy."

RDBMS are the root cause.

I feel like you are coming from an alternate universe. Nosql is the quick and easy thing to start with, then your needs become more and more complex and nosql just won't cut it anymore. Sure, nosql can scale and perform, but only if your needs are very specific and simple.

There are no major systems out there of even moderate complexity that aren't built on an rdbms.

> There are no major systems out there of even moderate complexity that aren't built on an rdbms.

I don't think this claim is accurate.

https://www.dynamodbguide.com/the-dynamo-paper/

That's the problem, business systems are not complex, they are incredibly simple, they are made complex by the way we think about and structure our models and then interact with them. We have the wrong boundaries, and the wrong languages

Would you be able to point me towards modelling approaches/boundaries/languages that would be more appropriate? I'd be interested to learn about better alternatives, as I don't yet see the big flaws in relational models

Unfortunately there is no real alternatives, its like operating systems, one or two system have so much momentum that using anything else becomes extremely difficult even when they are inferior in certain domains. see http://www.fixup.fi/misc/usenix-login-2015/login_oct15_02_ka...

A better alternative is having the data, data model and business logic tightly bound in one place. Not separated in multiple "tiers"

When one of these things change it changes the rest.

Datomic https://www.datomic.com/ a not-sql cloud database with a robust and principled information model competitive in use case with rdbms. In agreement with your parent, Datomic is very different and most commonly used from the Clojure programming language which rejects similarly much mainstream thought/practices. https://clojure.org/ https://clojure.org/about/state The talk Database as a Value https://www.infoq.com/presentations/Datomic-Database-Value/ is a good starting point to understand the flaws in relational model implementations (it has been argued on r/clojure that Datomic more faithfully implements relational algebra than RDBMS does – https://www.reddit.com/r/Clojure/comments/99f4ln/datomic_is_...)

Counter-intuitively Datomic is in violent agreement with /u/rqmedes where he said "A better alternative is having the data, data model and business logic tightly bound in one place. Not separated in multiple "tiers"" – Datomic inverts/unbundles the standard database architecture such that the cached database index values are distributed out and co-located with your application code such that database queries are amortized to local cost. Immutability in the database is how this is possible without sacrificing strong consistency, basically if git were a database you end up at Datomic.

Sorry I didn't explain it clearly. RDBMS are the reason we write business applications the way we do now. They are the root cause, swapping out RDBMS with NOSQL will solve nothing, because our languages, architectures, patterns, and libraries, how we even think about solving these problems all evolved on top of this and are flawed

I don't understand why this gets downvoted. While it may lack context (with the aim of being controversial), it sparked a healthy amount discussion here!

Reply on news.ycombinator.com