They're probably using the local ephemeral drives for Cassandra storage rather than S3. I'm guessing they're then moving snapshots into S3 or elsewhere.

http://www.mail-archive.com/user@cassandra.apache.org/msg110...