Hacker News Reader

Comment by mabbo | Hacker News Reader

The benefit: typically near infinite scalability with little loss of performance. It's super fast even when there's a lot of it.

The cost: transactions, relations. These are actually incredibly valuable things to have.

If you're storing billions and trillions of records that don't need relations, NoSQL is great. You'll keep your fast lookups and saves no matter how big things get.

This can matter a lot more if you're a start up with the goal of having trillions+ records some day. Your SQLServer will eventually not be able to scale any higher, and then what? Moving to Nonrelational from relational is so painful.

View on HN

I would say code for the scale you'll have in the near future, not for the scale you want to have in 5 years.

Transitioning database is painful, but dealing with the extra complexity of non-ACID stores can inhibit your growth. If you reach the scale where you need to change, you'll probably be able to afford more developers to help you.

Complete agreement. Like all of software developer, it's about tradeoffs.

infinite scalability with little loss of performance

I call shenanigans on that - Google's experience was that they tried it and then got bogged down as every developer had to roll their own mechanism for waiting for syncs - hence they developed F1 instead...

Near infinite scalability with little loss of performance? Definitely not with MongoDB. Can you show that with some numbers? I have seen quite the contrary: insertion speed slightly decreases with database size. Aggregate query speed decreases up to the point the database becomes unusable when you are on the order of small TB. Definitely a problem relational databases don't suffer (at least this significantly and at least at this scale).

I recommend you check out CockroachDB.

They're approach is to use a Key-Value DB to build a linear-scaling PostgreSQL-like Database.

Personally, I'm just waiting for that to become a bit more stable, but it's probably the most promising in this regard and already very usable.

"The cost: transactions, relations. These are actually incredibly valuable things to have"

This is quite arguable. Before the Computer Era, all businesses, including the whole financial sector did just fine without ACID. Still, ACID is used only at a local scope. I can often see ACID requirement used rather as an excuse for poor data modeling than a real need.

> This is quite arguable. Before the Computer Era, all businesses, including the whole financial sector did just fine without ACID.

Can you elaborate a bit? I believe that in the pre-ACID world, business processes were much slower, not online processes like today. When you're only really changing data once a day, backups and manual intervention are acceptable options.

> Still, ACID is used only at a local scope.

What do you mean by "local scope"?

ACID is not a way to deal with faster data change. Actually it is totally opposite. The more updates you have and the more online (distributed) you are, the less you want ACID. ACID is extremely latency-sensitive and failure-sensitive. And latency is a problem caused by just plain old physics - this isn't going to get better unless we find a way to make light travel faster than the speed of light ;) And the more distributed and larger your system is, the more frequent failures (like network partitions or servers going down) will happen.

As for "local scope" I meant "at a single business entity". If you look at a banking or online payment systems as a whole then they are not ACID, they are eventually consistent. The are based on two basic principles:

1. write everything down at least two times, so you never lose data

2. updates are incremental, so you never overwrite data

ACID sure is a way to deal with data change faster than once a day. Once you stop passing paper slips and order forms through your company, you can get accurate, up to date information at a human timescale, i.e. on the order of seconds.

Of course when you want stuff to happen at a faster-than-human timescale - or when you just want lots of stuff to happen - an ACID model might not be the right fit.

> As for "local scope" I meant "at a single business entity".

Understood.

My whole point is that ACID is not the only kind of consistency and it is not required to do "get accurate, up to date information at a human timescale". It is just a nice and convenient programming model, but it comes at a cost of availability, scalability, latency and throughput. You can design systems with no ACID data store and if you do it properly they can be just as accurate and up-to-date when everything works properly, but can also degrade nicely when some components fail.

http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on...

http://www.enterpriseintegrationpatterns.com/docs/IEEE_Softw...

> Before the Computer Era, all businesses, including the whole financial sector did just fine without ACID

The standard of the time looks quite ACID-y to me:

https://en.wikipedia.org/wiki/Double-entry_bookkeeping_syste...

Maybe it is similar, but you can implement it in probably any NoSQL store you wish. And it is not just a standard of the time, it is in wide use today, even if the underlyind database system is ACID.

Reply on news.ycombinator.com