Justin.tv's Live Video Broadcasting Architecture | Hacker News Reader

Justin.tv's Live Video Broadcasting Architecture

highscalability.com

58 pointsmattyb12 comments

I just compared that 45 gbit / sec with Amazon pricing. If it was hosted on EC2, bandwidth alone would cost ~$1.2M / month.

On top of that, the article says they've got settlement-free peering agreements with many top tier ISPs, which is almost a requirement for a bandwidth-intensive service like this. I'm sure it still costs them in the neighborhood of $500k/mo after all is said and done to run this backbone with all the equipment, space, power, telco fees, etc. It's rather impressive they pull this off with a staff of 32 according to TechCrunch.

I posted this comment to the article as well but I figure I might get more discussion of it here:

What exactly does "30 hours per minute of video is added each day." mean in layman's terms - 30 hours of video are uploaded each day? 30 hours of video are uploaded each minute of the day? I'm having trouble deciphering this metric.

On average, every minute of the day, a total of 30 hours of video will be uploaded.

Since 30 hours is 1800 * one minute, the above is a fancy way of saying we have an average of 1800 live channels at any given moment.

I understood that, in total, the uploads from all the users in parallel every minute comes out to 30 hours. A bit like saying YouTube serves n hours of video every minute because there are so many videos being watched simultaneously.

"Database schema upgrades are done by hand."

Cool, I thought I was just a total newb for doing schema upgrades by hand! (Well, I am a total newb but anyway)

I don't know what they mean by "by hand," but IMO, you should always script database changes and run the scripts in staging environments so you know you didn't forget anything and then it'll correctly run in prod.

We have a well-hacked version of ActiveRecord migrations that works with our replicated databases. We also have a number of different staging environments that we test with before deploying changes to production.

I think I've read every post on highscalability.com, and this is by far the most deep and detailed writeup they've done. Lots of insight into the huge scale of justin.tv and what it takes to push that kind of traffic.

Highlight for me was discovering twicecache. Love the narrow scope and "fit" for the job it's intended for.

Twice is great for us, but it's one of those things that evolved to meet our specific needs and architecture. If I were starting a new Rails app tomorrow, I'd start with Varnish, and only move away from that if absolutely necessary.

what is the story behind Twisted being phased out?

nevermind, I see it's all in Twice now :)

Reply on news.ycombinator.com