Hacker News Reader

Comment by bdcravens | Hacker News Reader

> The meaning of "joins don't scale" is that they don't scale across database servers, when your dataset is too big to fit in a single database instance.

You're correct. However, most applications databases are fine with billions of records on a single instance. It's a rare application that needs the level of scale you're talking about. In other words, a typical startup doesn't, at least not at first ....

> But if you're absolutely sure you need to be planning to run at close to Twitter or Facebook scale one day

Worth noting that both Twitter and Facebook started with MySQL.

https://dzone.com/articles/facebooks-user-db-is-it-sql-or-no...

http://highscalability.com/scaling-twitter-making-twitter-10...

View on HN

> Worth noting that both Twitter and Facebook started with MySQL.

And both are still using it as primary data store.

According to Stanford prof who launched ramcloud Facebook indeed uses MySQL. But MySQL by itself was hopelessly slow. That's why Facebook has over 4000 memcache servers (this as of 2009). By the time you add ram underlying disks he estimated 75pct of all Facebook data is cached in memory. That's why it works.

Instead of Stanford as source here is an example from the FB engineering blog about migrating from HBase to MySQl for messaged in order to reach performance: https://engineering.fb.com/core-data/migrating-messenger-sto...

And yes, they certainly cache data in multiple layers using different technologies but from all I as an outsider can see MySQL still seems to be their source of truth for most services. Meaning all writes go to MySQL and at least when going deep in history or doing searches (which typically can't be served from a cache) hit MySQL.

memcached + rdbms is really super cheap and effective. Properly architected it can be tolerant of all sorts of latency/failure/CAP issues, up to some limits of course.

there was a variant library that allowed you to replace memcached servers on the fly without worrying about the size of the cluster list or having to recache.

Define primary? Are you claiming every tweet lives in MySQL?

http://highscalability.com/blog/2011/12/19/how-twitter-store...

that's nearly a decade old

Reply on news.ycombinator.com