Adding WebRTC support to OBS using Rust | Hacker News Reader

Adding WebRTC support to OBS using Rust

github.com

261 pointsmarkdog1234 comments

Minor OBS tangent: as of macOS Ventura, OBS is now an extremely good screen recorder for macOS with much less UI jank: previously there were a few issues such as not being able to capture application sound or not showing the mouse, but now everything works well.

Incidentially if this WebRTC support takes off, it may be a great tool for high-quality peer-to-peer desktop streaming.

Did macOS Ventura help this somehow?

Yes. A point release added an API to do this better and Apple has contributed a PR to OBS to make use of the new API:

https://news.ycombinator.com/item?id=30112595

Yes. Also worth noting that for OBS 29 (in beta currently), Apple has contributed native hardware encoder support for H265 and ProRes, which both work very well. Very nice pick for their engineering outreach people!

edit: Forgot to mention that they also contributed Continuity Camera and Desk Cam support, so those are built in to OBS now as well.

Is there a way to confirm if the encoding is H265? I use the Apple Hardware VT Encoder and it doesn't provide many options (I use a Mac w/ a T2 chip which should be able to do H265)

If you're on OBS 29, all options will appear in the encoder dropdown in advanced output mode. The full suite: https://puu.sh/Juc1K/33d8aecc33.png

Ah, those aren't visible to me so maybe the T2 isn't enough and you do need AS. A bit of a bummer :(

They’re only part of the Apple Silicon hardware so that would make sense (though a Mac Pro with an Afterburner card would have it too, but I’m too poor to know if that works)

What are some advantages of OBS over, say, the screen recording tool built into QuickTime?

I haven't used QuickTime, but OBS does allow you to record multiple things and overlay them in any way that you please. Which is why streamers typically like it. The "share window + camera overlay" is common, but you can also add templates, multiple video streams, etc. Can you do this in QuickTime? I know most basic screen recording tools you can't do this so easily.

Since a couple years ago you don't even need to open QuickTime for a screen recording, just ⇧⌘5 to bring up the system native screen recording tool. If you just want a quick screen recording and nothing more, it's a lot more convenient than OBS, especially if you want to record a window or part of the screen. In OBS you need to manually configure output dimensions and cropping is difficult and unintuitive, you can't just drag out a rectangle on your screen. I don't know what gp was comparing to when they said "an extremely good screen recorder for macOS with much less UI jank"; the native screen recording tool certainly doesn't have any UI jank and is lot more user friendly.

Where OBS shines are

- Precise control over encoding parameters;

- Mix and match multiple input sources, beyond just screen + system sound + mic; this includes rearranging and cropping multiple application windows;

- Add text, images, filters, etc.

In short, it has a lot more power user features. If you just want to quickly record the entire screen / an application window / part of the screen, native screen recording tool beats it handily.

I was literally typing up this message when I saw your comment. I assume that the answer is yes?

> I'm pretty naive, but would this enable one to self stream to a limited audience? Like if I wanted to just steam my desktop to <5 people. I've kinda been surprised that there's a lack of tools that can accomplish this.

It would be really cool if I could present something from my desktop, show my face, and whatever, while just handing my friends a link and not requiring them to download anything.

I don't know the answer to the WebRTC part, but as long as you have a server with not-outrageously-priced outbound bandwidth, you can install an open source RTMP server like SRS[1], and stream to that RTMP server from OBS. It's really easy, configure the RTMP server & stream key, then "Start Streaming" which is right next to "Start Recording". You can then hand your friends a link, and they can play it in any media player with RTMP/HLS/FLV stream support, or you can add a simple web UI with e.g. hls.js[2] (very easy to write, there might even be prepackaged solutions) so that they truly don't need to download anything.

[1] https://github.com/ossrs/srs

[2] https://github.com/video-dev/hls.js/

I’ve always just used nginx rtmp:

https://github.com/arut/nginx-rtmp-module

The hardest part to establish peer to peer streaming like this is actually making sure all participants can see your IP address. Residential and cellular network are often behind CGNAT which make peer to peer connections almost impossible without using dedicated relay servers (e.g. SRS like other mentioned). Even if you have a publicly routable IP address, your router configuration might cause issues with peer to peer connections and it's non-trivial for average layperson to fix by themselves.

Love to see more WebRTC! Shame to see FTL go, I contributed a few PRs to Lightspeed[1] which relied on FTL to achieve sub-second latencies.

On a slightly off-topic note, I’ve been working on a simple WebRTC radio project[2] for a social listening experience. In my limited testing with a few friends, I was able to get < 100ms of lag (audio de-sync) between different players on different networks. It has been an absolute joy to use. The social experience of listening to music together somehow really appeals to me.

It was pleasantly simple to get it up and running with socket.io + Mediasoup as the SFU. I plan to flesh it out a lot more shortly but I’m a bit of a novice. Would love to have some more experienced eyes on the project :D

[1]: https://github.com/GRVYDEV/Lightspeed-ingest

[2]: https://github.com/obviyus/radio

Big fan of FTL also, it is a shame that it was orphaned. In a world where Mixer lives on I could see FTL being a lot more places. ftl-sdk was nice to work with since it was just a single C library, and only concerned itself with streaming client/server.

The way I see it WebRTC is the best way forward to enable people to build interesting things. I was so excited to itemize all the use cases that would be added to OBS by merging WebRTC support. I also wrote a little bit about 'Why WebRTC' https://pion.ly/blog/why-webrtc/. The big advantage WebRTC has over FTL is that is does more. Broadcast is a small part of WebRTC. I think WebRTC's superpower is that it can be used for so many different use cases and industries. Even if another protocol is better at a single use case, it is hard to beat the breadth of WebRTC.

WebRTC isn't without its issues. I think that is fixed by better education + more implementations and owners. We shall see what the future holds :)

A few years back I also used WebRTC with different backends (Kurento, Puppeteer, Mediasoup) to try and make an audio-based social network. I got pretty far but I was on my own and a grad student at the same time so the project died. A few years later Clubhouse became viral, it was essentially the same thing I was doing.

I still use WebRTC for my personal projects but the difficulty of compiling and embedding it in a backend makes it really hard to use by the masses. My next project is going to use Pion, and I'm going to make a static compilation and just distribute the executable and have the main service use IPC to communicate with the Pion module.

That sounds like a really interesting project!

If there is anything I can do to help please tell me, always happy to help :) Either email me sean @ pion.ly or join https://pion.ly/slack.

Slack is better, but some prefer email!

OBS is heavily architected around plugins, many of the built-in features including FTL output are plugins, so even if it is removed it would likely be possible to turn FTL support into a separate plugin that users can install. For reference the FTL code is in the plugin called obs-outputs (part of the main OBS repo). The only real limitation I know of is that it is likely not possible to make the UI exactly the same as it currently is.

> Co-authored-by: Sean DuBois

Sean is everywhere that involve WebRTC ! He is doing an incredible job.

The other webrtc allstar is Feross. Some of my earliest uses of the tech came from his very simple and effortless implementation of sending files to someone else simply with a link.

Instant.io

I had extensive comments yesterday praising this (especially the low latency), & calling out the future horizons (simulcast, multi-streaming) it might lead towards.

It's still questionable to me to call this WebRTC though. My understanding is it's only output is WHIP, WebRTC-HTTP Ingestion Protocol, which is used by CDNs to actually do the WebRTC protocol browsers do. It's near WebRTC but a couple steps short, seemingly. https://www.ietf.org/archive/id/draft-ietf-wish-whip-05.txt

Sorry I missed your comment yesterday! So many things got called out in the PR that I completely forgot about HN :/ You should re-post it in this thread! I really want people not familiar with see what is possible.

Agree on renaming from `webrtc_output` -> `whip_output`! I believe that is done in the current PR?

Copying rektide's comment here. I think it hits some really interesting points

-----

This is my vote for PR of the year in open source! Holy shit. There's amazing thing after amazing tbing listed.

> Latency. With WebRTC we are able to achieve sub second latency. With our initial measurements we see ~120 Milliseconds from Broadcaster to Playback. We believe we can continue to bring down this number as well.

More speculative, but awesome futures here that OBS will either adopt or eventually fall on the wayside to a worthy competitor on:

> Broadcast at all angles. WebRTC supports multiple video streams in the same session. OBS could experiment with broadcasting multiple scenes at once! Viewers could then switch between scenes or different views in the same game.

> Simulcast. WebRTC allows broadcasters to upload multiple streams of different quality. OBS users could upload high, med, and low streams themselves.*

Also notable Rust based, working via FFI. Is this the first Rust in OBS?

Im trying to understand a little deeper how this implementation works. There's seemingly some hardware encoding support, but as compared to using something general & flexible like gstreamer it seems a bit more hardcoded, and embedded in webrtc-rs. Not the worst I guess.

It also has this somewhat odd "whip" output, where it dumps the stream via http. Im not sure if that is the only output right now. This targets some CDN systems, which then host/peer over actual CDN. Calling this webrtc feels a little weird but ok.

Surprised nobody has mentioned the excellent https://vdo.ninja/

WebRTC is one of a very few newer browser features that has made me say "That seems legitimately useful!". Good to see it being integrated with more things.

I see a description for why webrtc-rs was picked and it seems to center around the impact to build time and size. It would be helpful to know what other factors were considered.

>Why webrtc-rs as the WebRTC library?

> WebRTC has many libraries that we could have used. The first one we evaluated was Google’s implementation known as libwebrtc. The time required to fetch+download would have been too much of a burden for the OBS code base. These metrics come from a Macbook M1 with a 400 Mb/s connection.

I think this only makes sense if they at least focus and get the components right that add the most value first, specifically P2P streaming. All the rest sounds like an advertisement, but I know it is open source and so I appreciate all the work.

This is very exciting. Latency can be problematic with OBS out of the box.

I know someone who did a full-fledged WebRTC plugin for OBS years ago for internal use only. Quite a shame it was never published.

Reply on news.ycombinator.com