Hacker News Reader

I use PostgreSQL regularly and pull (and even search) JSON blobs. We have something like 150 million JSON blobs (dataset is 500Gb) that are searched almost every query, response time is maybe 150-200ms... That's on an AWS t3.medium RDS instance.

I don't really know what the hell is going on with this, but something isn't correct.

View on HN

If its mostly on disk, you're paying ms for block io; get more iops.

If the index used are sensitive to low key diversity, fix that, and maybe shard.

You might prefer a vm type with nvme.

If u can divide the data into more small ones, more indexes and cache fits in ram.

Also, cluster data by what works, time, region, userclass, etc.

Only decompress the blobs at point of use.

Design your app to need only small fast data which cant be cached.

Etc

Reply on news.ycombinator.com