Hacker News Reader

Comment by jonknee | Hacker News Reader

1) Not everyone has their account at max 2) They get bulk discounts from Amazon that take the price down quite a bit 3) They only store one copy of a file across their user base, so even though they bill you for 100GB and you have it completely full, it's possible they are only storing a small fraction of that. A folder of music for example is likely to not actually take up any space at all, but they get to bill you for every bit.

View on HN

Except that the metadata on your MP3's is likely different, whether you've purchased them or just edited the genre in ID3 data.

I've often wondered if backup services "parse" MP3's into data and metadata for backup purposes, just because it can enhance deduplication so much. I've never heard of them doing it, but it's the first thing I would do...

Perhaps, but not a lot of people rip their own anymore. There are a lot of identical songs out there thanks to Amazon MP3, iTunes Music Store and all the various scene releases.

Whilst I don't disagree that few people are ripping their own stuff anymore, im pretty sure iTMS still puts the purchasers name in the metadata even now theyre drm free.

I can't recall of it was Dropbox or one of the other services, but I read they blocked up the files into blocks of a certain size. As long as the MP3 header doesn't change size (they usually contain padding or are at the end of the file to avoid this) it would just be one block that changed.

You don't need to worry about the particular structure of the files if you break the files into fixed-size chunks (e.g. 512KB) and do the dedup based on the chunks. The difference in the metadata of MP3 will be just in the first chunk, and the rest will be exactly the same no matter how you change the ID3 data. This strategy works for many other types of media files, too.

Reply on news.ycombinator.com