HNHacker News
TopNewBestAskShowJobs

nee1r

351 karma · joined April 18, 2020

https://neelr.dev hi! nice to see you~
submissionscomments
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
Around 2-5 hours/month, mostly powercycling the servers and replacing hard drives
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
We reached out to almost every colocation space in SF/some in Fremont to get quotes. There wasn't a difference between the quote price and what we ended up paying, though we did negotiate terms + one-time costs.
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
Thanks!! :)
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
Definitely much less redundancy, this was definitely a tradeoff we made for pretraining data and cost.
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
True, this is a large reason why we chose to have the datacenter a couple blocks away from the office.
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
We're currently 1/1 for the recent 4.3 magnitude earthquake (though if SF crumbles we might lose data)
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
Thanks! Lots of hard work went into them.
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
tips hat back
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
Already filled up and looking to possibly copy and paste :)
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
True, though this is specifically for pretraining data (S3 wouldn't sell us used disk + no DR storage).
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
We want to get darkfiber from the datacenter to the office. I love 100Gbps
nee1r··on Building the heap: racking 30 petabytes of hard drives for pretraining
yeah, exactly! we have a 100G uplink, and then we use nginx secure links that we then just curl from the machines using HTTP. (funnily HTTPS adds overhead so we just pre-sign URLs)
← PreviousPage 2 of 2