Hacker News Reader

Comment by Sohcahtoa82 | Hacker News Reader

> 188 TB of storage

> personal use

Who needs that much storage for personal use?

Anyone working with video, 3D animation, storing RAW photography (and taking 100's of photos on a daily basis), and I can imagine a few other purposes. Especially if you want an on-site backup of that data.

I've bought 10TB of storage space every year for the past 3 years and don't see myself stopping. I need the storage space! If these data-heavy hobbies were instead my daytime job I could easily see myself needing 100TB+ in storage (and then 100TB+ of backups). I fill roughly 8TB/year in storage space (4TB of data + 4TB backup data).

Some people believe in redundant backups. I only have redundant backups of extremely important information - which are mostly text documents so don't take up much space. But if someone was storing 3 copies of all of their media assets you end up using a lot of space. For example, 10TB of video turns into 30TB of video. And 180TB is really only 60TB of data, which isn't that much for data heavy hobbies.

Mostly Linux ISOs.

You wouldn't happen to frequent a certain, data focused, subreddit now would you?

All of them?

Someone shooting a lot of 4K or even HD video, for example.

From a little googling, 4k video appears to take about 22GB an hour[1], so 188TB would beover 8,500 hours of 4k video[2]. That's almost an entire year of constant 4k video. That's a lot of storage for personal use.

1: Not sure what encodig

2: Napkin calculation, corrections welcome

22 GB for an hour of 4k video is fairly compressed. Raw 4k footage runs closer to 300 GB hour.

Not just encoding, but it also depends on a number of things like color depth, chroma subsampling (4:2:2), and so on. It's not uncommon for a single film to approach half a petabyte of raw in a digital world with a "typical" setup (4:2:2, ProRes maybe, 10-bit, etc). To use an example, I think I read somewhere that Gone Girl shot a few hundred TB for a technically straightforward film. Some cameras, particularly when you start getting to digital cinema (which is > "4K"), can shoot as much as a terabyte and a half per hour. There are 8K sensors in common availability now; RED has one that can shoot 75fps 8K at 2.4:1, which I don't even want to calculate, but that I know for a fact it can't write to its own storage at native bitrate. Productions eat storage these days and moving that data around is a challenge.

Hint, hint, to clever founders: hard drives in Hollywood.

I would haircut the number for RAID then probably half it if he uses a backup.

Editing also tends to create a lot of huge temporary and intermediate files that will eat up disk space in a hurry. 188TB is a lot of space, but nowhere near unreasonable for someone doing video work. Plus, as other people have mentioned you need to account for filesystem overhead, redundancy (RAID), backups, TB vs. TiB, and so on.

Don't forget parity drives for availability take away from usable storage.

NAS #1 is 12 x 4TB with 2 drives used for parity.

NAS #2 is 20 x 6TB with 4 drives used for parity.

That takes away roughly 32TB of usable space under zfs. NAS #1 has 40TB usable. NAS #2 has 96TB usable. Total 136TB usable.

The remainder of the drives not accounted in the list above are in workstations for local storage and other random tasks.

My 1440p 4:4:4 videogame captures at 60 fps come out to about 700 GB per hour, sometimes more depending on the scene complexity. That would fill up a 200 TB array in no time.

I'd record to ProRes or DNxHR if I could, but there's no viable method for doing that on Windows without using an FFMPEG pipeline (which won't do 4:4:4 color sampling with those codecs and simply uses too much CPU at 4:2:2).

Why do you do that?

> From a little googling, 4k video appears to take about 22GB an hour

That's "home delivery" video, not the stuff you work on which would be far less compressed or even uncompressed, and would include reams of data thrown out entirely at the final production stages.

Think about the difference between "work" (mixing/mastering/production) audio data (uncompressed and at 24 or 32 bits) versus "consumer" audio (mp3/aac at 48/16).

Reply on news.ycombinator.com