Show HN: Overpass – a self-hosted video live streaming app | Hacker News Reader

Show HN: Overpass – a self-hosted video live streaming app

github.com

160 pointsGOATS-34 comments

Hello HN!

Overpass is an app I made for self-hosted video live streaming. I originally made the tool for a friend of mine's Patreon service, as he wished to have a private streaming service just for his subscribers. Unlisted YouTube streams were taken down for copyright issues when he did DJ sets, which became a hassle after a while. I originally started with barebones nginx-rtmp, but found it to be too tedious to ask people to open VLC or whatever media player they have to watch the stream.

From that point on, I decided to fully commit to making a live streaming service from "scratch". Overpass is my first venture into making a web application with HTML/CSS, as I've primarily been making CLI apps for my own personal use. It doesn't look all that fancy, but it does the job for us.

Overpass uses Oauth2 via Discord for authentication, as most people subscribed to his Patreon already have access to our server. This is why Overpass can also prevent anyone outside a certain Discord server from gaining access to the app.

There is currently no admin panel, and everyone has the same rights, so it is not intended for deployment in a large public community just yet.

It also automatically records and archives every stream!

The app is written in Python with Flask, however, I am planning to port it to FastAPI as I want to try something else.

I hope you guys find it easy enough to play around with, if not, let me know what I can improve upon. Thank you for reading!

I am sorry for being this guy!!! but when I see overpass and self-hosted I instantly think of hosting my own Openstreetmap server. This project looks very cool, and I know how hard naming things can be!

No worries, I named it Overpass as in the beginning the project was meant to be a very simple bridge between a web app and nginx-rtmp. I suppose you can see where I got the name from. :)

Most important thing for me is the latency of the streaming solution. Discord seem to do really well here, with <1 second latency times. You done any measurements with your solution to suggest how much latency there is between the server and the client?

If you're connecting to the RTMP server directly, you can get about 1-2s latency. However, due to the additional overhead of the conversion process to HLS, I'm getting about 6-10 seconds of latency.

I have considered looking into WebRTC, but it seems a bit more complicated to work with than just running an RTMP server and calling it a day.

Both Apple and Twitch have their own implementation of low latency HLS, so it could be worth looking into how feasible it would be to setup something similar.

https://developer.apple.com/documentation/http_live_streamin...

Lowering hls_fragment[1] to 1s should help out a little with the latency. This is what I've done in the past with a similar project as yours, Open Streaming Platform[2], with good results.

Since you mentioned WebRTC for streaming to provide even lower latency, this is what I believe Project Lightspeed has achieved[3]. Might be something you'd be interested in, if you hadn't already seen it.

[1] https://wiki.openstreamingplatform.com/Usage/Streaming#osp-n...

[2] https://wiki.openstreamingplatform.com/Usage/Streaming#osp-n...

[3] https://github.com/GRVYDEV/Project-Lightspeed

I've experimented a bit with lowering the hls_fragment value and the playlist length, and found that fragments smaller than 2 seconds caused too much buffering on the end client for it to be stable - which resulted in higher latency than before.

I'm looking into the latency issues and I've started drafting up possible solutions. Low latency MPEG-DASH might be something to experiment with. It won't reach sub-second latency, but it will definitely be better than HLS.

This looks great and I hope to deploy it and kick the tires soon, thanks for sharing.

Currently I've been using movienight[1] for this, but it sounds like your app is much more feature rich.

[1] https://github.com/zorchenhimer/MovieNight

Why can't your friend use Discord streaming? Don't think there's copyright takedowns,

The quality isn't good enough. We also wanted an archive of every stream we've done.

Curious if you considered Mux: https://mux.com/live or IVS: https://aws.amazon.com/ivs/ for your project? If so what made them not compelling options?

I think the main reason was that I already have a dedicated server, so it made sense to use the already existing infrastructure for cost saving purposes.

I've also looked at Cloudflare and BunnyCDN's options. Could be worth thinking about if I ever move away from owning my current server.

This is pretty well done! Coincidentally, I've been working on a very similar project for a friend of mine, for the exact same reasons. Self-hosted live streaming is definitely the way to go (~8% costs versus ~50% Twitch fees), so it's great to see more independent and open source solutions popping up!

I do think it's important to note, however, that self-hosting doesn't legally absolve the streamer from PRO fees for playing covers, and they'll still need to register and pay based on actual revenue. PROs are notoriously aggressive about pursuing that kind of skirting.

I wonder how do you calculate 8% costs. Also is it possible to include in the calculation the "discovery" service that twitch provides?

Twitch discovery is not what they pretend it is. A raid results in a tiny amount of follows, and very few people actually peruse the browse page.

As for the calculation, that's based on: 2.9% + 30c card processing, ~3% PRO fees, plus bandwidth and server costs.

Twitch has 0 costs, and provide you with avenues to earn revenue (ads, bits, subs).

Your self-hosted platform has some costs, and doesn't have any way to directly monetize via ads or similar.

Twitch takes 40-60% of everything, so that's not "0 costs." Bits are just donations, and every streamer already tries to get people to donate directly to circumvent Twitch's fees, and subs are possible through Stripe directly, or through Patreon or their self-hosted version.

So, literally none of what you said is reality.

Twitch takes a cut, but doesn't "cost" anything.

When you signup and decide to start streaming, do you have to buy a subscription or pay for something?

You're making some ridiculous assertions that do not reflect reality in any way. If a business pays a commission to a service it utilizes, that is a cost. If the viewer is paying $5 and Twitch takes $2.50, it COSTS me $2.50 to participate in their system.

Whereas, what you're suggesting is that the expense magically doesn't even exist, simply because you didn't need to pay it up front. Except that's not how anything works.

This looks badass. I have been thinking about building something just like this for similar reasons.

One thing I was considering was using cloudflare workers and durable objects to build the important part. I.e. the fan out of raw video content from producers to consumers.

I would also consider building an integrated and LDAP/SAML authentication option for the enterprise licensing tier...

Very interesting! Can you tell me about that Cloudflare Workers setup? Would you have some sort of worker process the ingress video?

Very new to the concept of serverless computing, so I would love to learn more.

The workers would handle both ingress and egress. The ingress feed should already be encoded for distribution by whatever system is pushing it. The durable objects in the middle would hold a moderate buffer of content which would be referred to on the egress routes. Archival would be a separate system that listens to all egress sessions at the same time. A 3rd websocket CF worker route could be used to exclusively manage video player/streamer/editor events.

One low-latency system I had in mind would go: individual streamer frames (jpeg) -> PUT CF route -> route adds durable object per frame -> GET CF route w/ current frame counter returns newer frames -> Client web UI displays the jpeg frames with appropriate timing via a 2-3 frame buffer.

This has much higher bandwidth requirements due to the loss of interframe compression effects, but it is compatible with 100% of platforms and very fast. It also allows for very simple seeking.

If you are willing to leverage cloudflare and your target audience can handle a ~25 mbps download requirement, then this model could fit. You can get more mileage per bps if you reduce quality/resolution too.

You can also put some intelligence in the stream path. Distributed compositing pipelines and other fun ideas. It's infinitely easier when everything is natively "just an image". Maybe some server's only job is to draw the hud on top of a frame received from a datacenter across the street.

so you're going to playback mjpeg to people? I don't think this is going to work very well. at the very least it needs to be a shipping video in the form of a remotely modern codec/container to people.

> I don't think this is going to work very well.

I've built prototypes using libjpegturbo for encoding to web clients and it works out fantastically in my experience.

I don't see a requirement to ship anything remotely modern if the desired product experience is achieved regardless. No one said it always has to work on every class of device everywhere for every type of user. Maybe someone just wants something that functions in any reasonable web browser and isnt interested in directly competing with twitch.tv.

I mean, I get what you're going for but there's a reason why nobody does this. mjpeg is just about the most inefficient thing you could possibly do. Maybe it would be fine if the desired product experience is "make it look like an animated gif and take 40 gigabytes to store an hour of video"

edit: not trying to discourage! Just saying this is a pretty well-trodden path and mjpeg is mostly abandoned despite ubiquitous support for it

> the jpeg frames with appropriate timing via a 2-3 frame buffer

Oh people have tried this many times. Don’t bother. Both x264 and NVENC can encode with no buffering (1 frame at a time) and get the same latency benefits without killing quality and bandwidth. JPEGs just look terrible unless they’re lossless.

That aside, you brought up the real issue with open source DASH client implementations: They have terrible buffering algorithms that keep several seconds of video in memory instead of 2-3 frames even though the user might have a decent connection. Oh and by default they use the HTTP Date header for time synchronization between client/server so you only get 1 second resolution lol

Part of the issue is that the majority of nontechnical users are on a shitty connection (WiFi or mobile data). They’ll always have to buffer >1 second (>60 frames) due to packet loss so they won’t see any benefits from a better implementation.

I need this at work, but with very low latency.

Teams does low latency well enough, but the frame rate is ~3-5 FPS which is abysmal when you're trying to follow someone's mouse cursor to see what they're actually doing.

anyone know of anything?

MS acquired Peer5 for Teams recently so I imagine improvement on the horizon.

I built an nginx-rtmp oss project a few years ago and it worked well—simulcasted out to Twitch and YouTube and saw comparable latency (YouTube: 3 2 1 ispooge live is faster). I used a dedicated encoder box, recorded and proxied via my raspi3 and served from home fiber behind CloudFlare CDN with caching to prevent much origin traffic, 2sec HLS chunks were like 1-2Mb at 720p30.

YC company: https://www.daily.co/

oh, uh, I forgot to mention: I need it to be self-hosted entirely.

otherwise this looks very promising.

thank you.

ah, screego is probably what I'm after.

Hello GOATS, Very Nice, i want to share also these 2 FOSS projects i found, that can be a good alternative to discord, if can be useful:

MiroTalk: https://github.com/miroslavpejic85/mirotalksfu & https://github.com/miroslavpejic85/mirotalk

Keep it up!

This is great! Where are the archives stored? Locally on the streamer’s machine or on a remote server?

On the server Overpass is hosted on. I'm considering moving to some sort of object storage when Cloudflare officially releases R2. I'm also looking at deploying my own instance to Fly.io at some point in the future.

Really cool stuff.

Reply on news.ycombinator.com