Sean-Der

What platforms were you targeting that you found it painful! Sorry it was frustrating.

I hope it’s getting better with education/more libraries. It’s also amazing how easy Codex etc… can burn through it now

I am excited for VAD to go away. PersonaPlex totally seems like the future.

However things like 'Call center helpline' turn based actually seems better! You don't want to be interrupted when giving information back and forth (I think?)

Thanks for reading it!

You can't beat Websockets :) Especially since you have so much tooling/existing stuff that works with HTTP.

I have been trying to get a website off the ground that does Datachannels + SQlite in the browser and then users sync between each other. I have gotten distracted so many times though.

Thanks for using it :)

I think It’s a case of you improve what you own. The owners of WebRTC servers were aggressively improving their part. They don’t own the inference servers.

Did you use libwebrtc on the backend? When you say `libwebrtc` is the only game in town are you talking about clients or servers?

Even for clients you have things like libpeer that libwebrtc can't hit.

Check out [0]. You can do 'Voice AI' on small/cheap hardware. It's the most fun you can have in the space ATM :) It's been a while, but posted a demo here [1]

[0] https://github.com/pipecat-ai/pipecat-esp32

[1] https://www.youtube.com/watch?v=6f0sUEUuruw

It doesn't today, but you could with sometime like this [0]. You can save/suspend all WebRTC state and bring it back with the next process.

[0] https://github.com/pion/webrtc-zero-downtime-restart

Very grateful that OpenAI published the article/publicized their usage of Pion[0] a library I work on. If you aren't familiar with WebRTC it's a super fun space. I work on a book WebRTC for the Curious [1] that details how it works.

[0] https://github.com/pion/webrtc

[1] https://webrtcforthecurious.com

What USB Tuners do people like? I have a Hauppauge WinTV-dualHD Dual Tuner - 955D and I have to restart server every 3 days because it deadlocks.

Do any alternatives to TvHeadEnd exist? I looked a bit and seems like it’s the best. So much customization it would be hard to reach it’s quality

> use WebRTC and deploy selective forwarding units, which are going to be something custom

Would you mind explaining more? If you are doing WHIP/WHEP you should be able to drop in Broadcast Box/MediaMTX etc... and switch out servers and no one should notice. You can use browser/mobile/ffmpeg/OBS etc... get the same behavior. I care a lot about the broadcast space, want to learn about other problems.

> subtly speed up audio/video to keep everything in sync

You can use https://webrtc.googlesource.com/src/+/refs/heads/main/docs/n... to add more delay (if you want to force more buffering). Or if you don't link the media together (via MediaStream) you don't get the behavior you describe either!

> capture each participant's audio individually

That's a neat problem. I haven't solved this one myself, I wonder if it's easier with RtpTransport or insertable streams?

Huge fan of BigBlueButton! It’s been cool to watch the project go through multiple big tech changes and still keep going.

Never give up! Free/Open Source software especially in schools is so important

What language? Would love to help :) especially with AI Coding I think it would be a lot more accessible these days.

ex_webrtc is super cool. They have a cool built-in dashboard/analytics flow. It is way more 'operations friendly' then Pion it seems. I haven't used it heavily myself though.

Yep exactly! After it starts you can slice in any media you want.

You need to make sure you are sending the same codec that the remote expects, otherwise nothing else! You can do a different resolution, bitrate etc...

Yea!

* Do video playback out of the browser. You can render a subset of frames, use a different pipeline for decode etc...

* Pull video from a different source. Join Google Meet on current computer, but stream from another host.

Have you tried doing video + pipewire yet?

I am also using v4l2loopback, but its annoying to juggle /dev/video* devices. I wanted to do video stuff in docker containers, and it would be amazing if I could do pipewire in each container and have no global state.

I couldn't get anything to work in Chromium. FireFox saw the device, but video didn't come across.

I can’t wait for https://w3c.github.io/webrtc-rtptransport/ when you talk about pulling vide out seems like the perfect fit.

I ended up doing proxy because Google Meet doesn’t let me hook at any RTCPeerConnection APIs at all. I wanted to send synthetic media in, but couldn’t get it working. Ending up doing a virtual webcam on Linux.

Oh yes! I will pull together a demo.

With ‘media-send’ I can send it out to ffmpeg/GStreamer and that does all the heavy lifting

Thanks :)

No better feeling to work on something and hear it is novel! So many projects that I think will be useful miss the mark.

I wrote this to make Reverse Engineering WebRTC services easier. Will also let you save/send arbitrary media from WebRTC sessions. The idea is you do all your auth/interaction in the browser, but then do all WebRTC in Go. So you have lots more control. More to do with it, but it is far enough along to share at least.

In the README is an screenshot of sending my webcam, but replacing outgoing video with a ffmpeg testsrc. Handoff sits between so it can replace with any arbitrary video.

SFU is nice because it gives everyone privacy. Neither you or ISP know who you are communicating with.

Otherwise if everything is trusted I love that we can have conversations without depending on others over the internet :)

What’s painful about running LiveKit? What would make running WebRTC server easier?

The ecosystem around RTMP is bonkers.

RTMP has SO many users for video (Twitch, YouTube etc...) yet you have librtmp which has so many forks. OBS has its own version in-tree....

Then at the same time you have people trying to add extensions to RTMP still!

RTMP has held back the use cases I have cared about since ~2015 so I am excited to see people embrace other options.

That's exciting! When you were evaluating it everything about the protocol/APIs fits your needs?

Just features/software need to be implemented?

What was the WebRTC bug, would love to help! I saw at work that FireFox doesn't properly implement [0] I wanted to go fix after FFmpeg + WHEP.

If you are still struggling with WebRTC problems would love to help. Pion has a Discord and https://webrtcforthecurious.com helps a bit to understand the underlying stuff, makes it easier to debug.

[0] https://datatracker.ietf.org/doc/html/rfc8445#section-7.2.5....

I really thought that OBS + WebRTC would be a few months project. Ended up being way stickier than I expected.

Most days I’m more excited for this space than Pion. It’s just more fun to make software for users vs businesses mostly doing AI maybe?

If anyone is using/testing WebRTC I would love to hear how it is working for them :) I am hoping Simulcast makes a impact with smaller streamers/site operators.

* Cheaper servers. More competition and I want to see people running their own servers.

* Better video quality. Encoding from source is going to be better then transcoding.

* No more bad servers. Send video to your audience and server isn't able to do modification/surveillance with E2E Encryption via WebRTC

* Better Latency. No more time lost transcoding. I love low latency streaming where people are connected to community. Not just blasting one-way video.

If anyone is using/testing WebRTC I would love to hear how it is working for them :) I am hoping Simulcast makes a impact with smaller streamers/site operators.

* Cheaper servers. More competition and I want to see people running their own servers.

* Better video quality. Encoding from source is going to be better then transcoding.

* No more bad servers. Send video to your audience and server isn't able to do modification/surveillance with E2E Encryption via WebRTC

* Better Latency. No more time lost transcoding. I love low latency streaming where people are connected to community. Not just blasting one-way video.

I built a Open Source tool to help people debug/understand their live stream quality better. I would really appreciate people's feedback! https://streamsniff.com/

I see a constant stream of people saying 'my video is blurry' or 'why do I have so much latency' and I wanted to build something that helped people understand/diagnose. Also wanted to make something that runs in the browser so you can share/send a link to a friend (or on discord) to make debugging easier.

https://github.com/sean-der/stream-sniff is the code.

I wanted to help people debug/understand their live stream quality better. I would really appreciate people's feedback!

I see a constant stream of people saying 'my video is blurry' or 'why do I have so much latency' and I wanted to build something that helped people understand/diagnose. Also wanted to make something that runs in the browser so you can share/send a link to a friend (or on discord) to make debugging easier.

I am hoping to move it to https://github.com/glimesh org soon so it can sit next to broadcast-box.

WebRTC is a real super power on this stuff :)

I also love seeing it used for 'kill the jump box' and file transfer. Just drives me crazy that we lets files sit on file providers.

Especially if you are transferring in the office! Send it right over the LAN and could be instant. Being forced to upload + download from remote servers frustrates me.