> We at ClubNFT fired up a massive AWS instance with 40TB of EBS disk space to attempt to download this, with a cost estimate of $10k-20k over the next month,
Is this for real? Why would they do it this way? I can buy 20TB of storage for less than $1000 (in New Zealand, no doubt far far cheaper in USA). Using one fifth of my residential gigabit bandwidth that would take just over a day to download.
Why on Earth would someone spend $20k and a month to achieve what can be done for one twentieth the cost in one thirtieth the time? Only to have the files on a remote AWS instance where you would still need to download them if you wanted to access them?