> but that's all built on the assumption the file exists on a normal file system with locks, consistency, etc. However, this is not the case here.
It is the case here. Writes are happening on a normal linux VPS, and subsequently copied to S3. The read-only queries are using the HTTP VFS. It would be incredibly bad to enable direct writes to a db from arbitrary clients.