Hacker News Reader

Comment by winrid | Hacker News Reader

Wow, never realized Pinterest had 6 petabytes of data. I wonder if they're including images in that. Even billions of rows is usually around 1-2tb, so makes you wonder what they're storing many billions of.

View on HN

They almost certainly have trillions of rows of data. The popular social media / user-generated content sites are just huge. You can easily get to that size just from the core product OLTP data/metadata, no need to include media.

For one comparison, Tumblr hit 100 billion unique rows of relational data in MySQL (on masters, not including replicas) back in October 2012. So they're easily in the trillions of rows today, and Tumblr is smaller than Pinterest!

Just surprising since I've never seen the social side or anyone using it. Guess I'm just not in that social bubble.

Woman use it a lot.

My wife has a Pinterest board.

Pinterest has over 500 million monthly active users. Their revenue was $3 billion in 2023.

It makes no sense at their scale to store images in Hbase. It's probably just user tracking/analytics.

In NoSQL world, you often store multiple duplicates of the same data, each of which is optimized for a different use case.

The SQL world calls this an "index."

I am attempting to reach enlightenment by contemplating your wisdom.

No. Most NoSQL dbs support indexes and secondary indexes too.

In SQL world we call this "denormalization". It's called the same thing in NoSQL, too.

It's called materialized views.

Well, yes, and no. :)

Indices are essentially managed denormalization.

Ah, never thought of it that way.

Indices are no good if you constantly do large amounts of inserts or updates (say, entire clickstream of a popular global site) - keeping them up to date will massively slow down the inserts/updates. Wheresas, in NoSQL, you can do as inserts as quickly as your disk can write data, and still query the dataset as if it had an index.

That has nothing to do with whether the DB is relational or not.

Which relational databases can simultaneously do milions of updates and milions of selects per second, to the same table?

If a company has HBase and doesn't have anything else, then the developers will store everything in HBase due to lack of alternatives. They'll store logs in HBase, metrics in HBase, images in HBase.

Reply on news.ycombinator.com