Hacker News Reader

Comment by malka | Hacker News Reader

DELETE is free LIST is not

I guess one could spam DELETE calls while bruteforcing filenames to make it free.

An S3 inventory report [1] is $0.0025 per million objects listed versus $0.005 per LIST request. Make an inventory report request, retrieve the inventory report from S3, make your free delete calls synchronously with API calls or pay for a batch job [2] if you'd rather not write code ($0.25 per batch job and $1.00 per million object operations performed).

It's listed in the post as "extra credit", but its trivial having done it myself for a client. A bit disingenuous to state, "if you like to do things the hard way." It literally takes less than hour to do, and you can preserve the inventory report for housekeeping if needed.

[1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/storag...

[2] https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-...

(OP: feel free to steal this comment's info if you want to update your post)

> It literally takes less than hour to do

Yikes. An hour just to delete some files in a folder? It seems to me that S3 should be avoided unless you really, really, really need it.

When the scale of data goes up, so does the complexity of managing it. Deleting 100 rows from a live psql table with 1000 entries is very different from deleting 100M rows from a live table with 1B entries.

S3 Batch Operations list of Supported Operations[0] does not include DeleteObject or DeleteObjects.

0. https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-...

I appreciate you pointing out my mistake. That wasn't clear from the documentation I referred to [1] (when I performed the operation, I created a queue from the inventory report and performed delete requests popping the items off the queue), and I've let our TAM know the documentation I referred to needs to be improved to reflect that DeleteObject or DeleteObjects operations with a batch job aren't currently supported.

[1] "Specify the operation that you want S3 Batch Operations to run against the objects in the manifest. Each operation type accepts parameters that are specific to that operation. This enables you to perform the same tasks as if you performed the operation one-by-one on each object."

https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-...

Reply on news.ycombinator.com