Hacker News Reader

Comment by throwaway91111 | Hacker News Reader

It's amazing how much you can put into a reply without directly addressing any of the concerns. Are you serious that dynamodb can't even hold 10GB? If you intended to defend the database I think you failed. Sqlite3 can store 10 gigs without sweating. What on earth can you use dynamo db for if it doesn't scale with the data? The way you describe it seems much WORSE than the OP.

View on HN

I said it was per node (which is a shard), it's not a limit per table. Did you read the original article or my reply? Both of them state this is per node and how DynamoDB decides when to shard to more nodes.

Edit: Sorry, I did mention 10GB originally in relation to a table. That was incorrect of course.

I did. However, 10GB still seems extremely small. A commodity postgres, cassandra, or cockroachdb server can serve HUNDREDS of GB per node. Why is the size per node so small for dynamodb?

It seems like poor key space design.

Agreed entirely, I do wonder what kind of internal constraints led Amazon to have these limits. Maybe in the future they'll go away, as so many other seemingly arbitrary limits do.

While not exactly the same, the Dynamo paper outlines that a single host is composed of many virtual nodes. It is very likely that a physical DynamoDB host will have dozens of nodes. This is done so that the cluster can scale up or down independent of the number of hosts while avoiding a gross imbalance. (12 nodes on 11 hosts means one host has 100% more traffic. 34 nodes on 11 hosts means one host has 33% more traffic.)

It is important to note that "Dynamo" and "DynamoDB" are two very different things that happen to share many of the same letters. DynamoDB is not Dynamo.

This Cassandra article should provide some info on virtual nodes. https://docs.datastax.com/en/cassandra/2.1/cassandra/archite...

10GB is per partition, not per node.

Reply on news.ycombinator.com