Hacker News Reader

Comment by amatecha | Hacker News Reader

Nice, do you have any summary of what process you take to back up channels? I've considered the same in the past, but found that there are a lot of crappy/sketchy plugins, and not too many legit/decent ones that actually get the highest quality copy of the content.

View on HN

youtube-dl is great for this, you can run it periodically against a channel and only download new videos. See the --download-archive option. Using "-f best" will fetch the highest quality format, but you can be more specific with filters like "-f best[filesize<50M]".

Recommended storage device and disk drives?

https://www.backblaze.com/blog/open-source-data-storage-serv... ; it's what I use to keep anything in cold storage that can't immediately go into the Internet Archive or that they're unable to dark after I upload.

Wait so you bought a $10k 60 disk storage platform? Or built one?

Built (several). My background in tech is ops/infrastructure/networking/devops.

That's awesome. I would like to build something similar, though maybe scaled down to 1/2 size/capacity. Did you follow their parts list exactly or deviate (e.g. what did you use for chassis since theirs is custom)?

> what did you use for chassis since theirs is custom

FWIW, in addition to their awesome disk stats Backblaze also provides blueprints for their chassis as well as a BOM for the parts inside

> Backblaze also provides blueprints for their chassis

Wow, I had no idea they did that. I definitely have a new respect for them.

What does "dark" mean in this context?

Publishing of content that would cause legal or other harm to the Internet Archive. It’s still stored on disk, but not accessible.

To clarify, to “dark” an object is to make it unavailable but continue to store it.

Really! I had no clue about this one (it has been a while since I searched around for such a thing).. that's great! Thanks for the tip, I'll check it out.

Yeah, I'm basically doing what you describe:

`youtube-dl -f best -citw -v <url of channel or playlist>`

If I recall, that's going to ignore errors. At an airport right now so I can't remember exactly what those flags are.

youtube-dl has this in their FAQ:

> Do I always have to pass -citw?

> By default, youtube-dl intends to have the best options (incidentally, if you have a convincing case that these should be different, please file an issue where you explain that ( https://yt-dl.org/bug )). Therefore, it is unnecessary and sometimes harmful to copy long option strings from webpages. In particular, the only option out of -citw that is regularly useful is -i.

But OK, here's explantion of the options you used:

• -f best: Select the best quality format represented by a single file with video and audio. (By default, yt-dl will merge best video with best audio if that's what's available.)

• -c: Force resume of partially downloaded files. (By default, yt-dl will resume downloads if possible.)

• -i: Continue on download errors, for example to skip unavailable videos in a playlist.

• -t: Use title in file name. (Deprecated. This is the default.)

• -w: Do not overwrite files.

• -v: Print various debugging information.

I actually use -f bestvideo+bestaudio, since it downloads and muxes the best combination of DASH video and audio.

ArchiveTeam also has a recommended list of flags for archival: https://www.archiveteam.org/index.php?title=YouTube

Thanks! I'll be doing this from now on. Had no idea about that.

>I actually use -f bestvideo+bestaudio,

It does that by default. (If you have ffmpeg)

I thought it already fetches the highest quality by default?

It has for a while.

I'll throw in my vote for youtube-dl as well. I love that it's so cross-platform; there's even a port for OpenBSD which makes it super easy to enjoy YT videos on that OS without having to run a resource-heavy browser, and it can be scripted.

You can also use Yizzy[0]. It’s just a GUI on top of youtube-dL but it’s very easy if you don’t like command line. It’s only MacOS compatible by default but should work anywhere if you change the python path. Like youtube-dl it also allows you to download whole playlists by just pasting the link.

0 - https://github.com/biko-the-bird/Yizzy

also https://github.com/MrS0m30n3/youtube-dl-gui

and there are plenty of others too.

Reply on news.ycombinator.com