> you can’t is because it’s unsafe to expire old data in such a system if any backup is writing
I did this, I just don't expire any object created in the past week.
I did this, I just don't expire any object created in the past week.
If a backup and a GC race then a file can get both referenced and marked at the same time, but then a future GC will see the references and put the file back into a normal state. Assume other operations can still find the file while it's marked.
Are there benefits to CAS for this situation other than resolving faster?
> Are there benefits to CAS for this situation other than resolving faster?
I think this kind of thing comes up a lot, where you’d find it convenient to have a CAS update for your file. Like, maybe you should be using a database, but you’re already using S3 and having one or two CAS operation would mean that you can stick with S3.
Sometimes, the alternative is a little ugly. Like, “I’m going to create a DynamoDB table, and it’s only going to contain one row.”
What I’d really love, even more, is to have some kind of distributed lock service on AWS. Something like Zookeeper or Etcd as a SaaS product, where it’s cheap just to get a couple distributed locks. Feels like a gap in cloud offerings to me, but I can understand why it’s missing.
LIST, GET, and PUT are strongly consistent, the file name is the lock name, write the owner id and expiry timestamp in the file, and periodically extend the lock expiry (heartbeat). If an other process finds an expired lock delete the file.
It's just that a backup tends to have mostly immutable files sitting around, so it becomes more niche. It's awkward to do a lock but you don't need a lot of locking.