Hacker News Reader

Comment by hansvm | Hacker News Reader

While we're joking about AWS adventures, has anyone here seriously contemplated using S3 keys as a cheap blob store? It looks like they're a generous kilobyte a pop, and prefix queries are supported, so you could hypothetically chunk your data into a [key|index|data_index| scheme and store a bunch of empty files in slightly under 1KB chunks -- ingress is nearly free (API calls, though you can batch them to keep this from being terrible), egress is expensive, and storage is free. Does that work?

View on HN

That is why you also pay per request :)

This is a terrible idea. S3 charges $0.005 per 1,000 PUT operations. So let's say you want to store 1GB (1e9 bytes) 1,000 bytes at a time, that's 1,000,000 keys.

$0.005 * (1,000,000 PUT operations / 1,000) = $5.00 to write 1GB. The on-going storage cost I believe would be free since the files themselves are empty.

Meanwhile writing a single 1GB file to S3? Costs you $0.023 per month storage, and the single write cost is $0.000005. It'll take 217 months to cost $5.00

Trivia: this API is I believe the only documented time that AWS has ever raised prices. It was done just for that reason.

If you track the extortion that AWS charges for egress bandwidth against both inflation and what bandwidth costs on average then they are always raising that rate.

They changed the pricing dimension when it was in beta.

They've also raised SNS prices in some markets where regulatory fees increased, boosted pricing for domain registration several times, redid the pricing model for IoT Core when it was in beta, and changed the pricing model for AWS Config in ways that are kinda... unfortunate, for a lot of us.

How do you know? Who are you suppose to be?

I’m kidding, point taken. I listen to your podcast all of the time.

You also called a project I was heavily involved in at AWS “This may be the worst AWS naming of 2021.”

I took it as a point of nerd pride. Out of all of our bad naming convention, that said a lot.

Isn't that partially mitigated via S3 Batch?

Not to my knowledge? Can you link me to the S3 Batch documentation around PUT'ing multiple files from an external source onto S3?

Having worked at AWS for about half a decade in the past, I can guarantee that if you've thought of a way to abuse^W leverage their services for alternative purposes, someone has thought of it internally, as well. I knew someone back in 2014 who was working on a project to build a data store on tags, since tags were free. (The reality is that the tagging infrastructure would never have supported this kind of large-scale use, though. I remember a customer who was applying multiple unique tags to every single EC2 instance they launched, and by the time they had launched hundreds of thousands of instances, they were seeing very real latency when interacting with tags. I think the underlying performance issue was resolved, but these sorts of gotchas are all over the place.)

Is it cheaper than just using route53?

Reply on news.ycombinator.com