If you have a web service counterpart you should consider looking into webrtc... the sdp exchange can happen through your site and then they will connect directly
That's a valid point, and something like this might be a solution (i.e. serving the client JS from a public site with valid TLS certificate and connecting to the local network from there) -- however, I don't want my application to be dependent on an internet connection. This is something you should be able to run in the literal lonely cabin in the woods.
You could use service workers to aggressively cache the client for offline use. And maybe have an insecure version served from the application as a fallback.
I think manually trusting a self-signed is the simplest way to go in that case.