VPN over SSH? The Socks Proxy | Hacker News Reader

VPN over SSH? The Socks Proxy

blog.gwlab.page

191 pointscorychu97 comments

I used this feature pretty often, but it has one downside: all connections are multiplexed into single one which is not good for performance.

So I've implemented own client which decouples connections from each other: https://github.com/Snawoot/rsp#performance

Basicly, you get working proxy with speed almost as native connection as soon as you have SSH access somewhere.

Can multiplexing be disabled via `sshd_config`?

No, it's the SSH client who responsible which SOCKS request will be wrapped in which SSH connection. Stock implementation opens forwarded connections inside virtual channel of single SSH session.

Besides that you have to keep pool of steady established SSH sessions in order to start new connection forwarding inside separate SSH session as soon as incoming SOCKS requests coming.

Plain SSH client is neither able to maintain multiple SSH carrier sessions nor keep reserve pool of steady underlying connections.

> Can multiplexing be disabled via `sshd_config`

The multiplexing you can disable on the server-side is a different multiplexing, but also another useful ssh feature.

If you have a workload that involves sort of firing off many commands over ssh one after the other (i.e next command is based on the output of the previous), then you can make them all grab a prenegotiated ssh connection to speed it up.

Basically, I almost always have

    ControlPath ~/.ssh/control-%r@%h:%p

    ControlPersist 1m

in my ssh configs to take advantage of this in my Makefiles which need to ssh across for various reasons. Ssh commands to a "central location" are great for initializing env vars with AWS keys, instead of encoding them in shell scripts - easiest way to prove your identity is to prove you have your private key.

Excellent! Just installed it. Works like a charm. Previously couldn't use socks5 over ssh for long, due to this problem.

This looks really useful. Gonna give it a try!

Hmm, cool project. Making it work over MPTCP [1] could also make it a kind of replacement for shadowsocks [2] for the purpose of converting plain TCP to Multipath TCP, as it is used in OpenMPTCPRouter [3]. Shadowsocks is used for MPTCP proxying instead of plain socks exactly b/c it uses separate connections for separate flows.

[1] https://www.multipath-tcp.org

[2] https://shadowsocks.org/en/index.html

[3] https://www.openmptcprouter.com

Or SCTP

Wow this is so cool. Sometimes, OpenVPN traffic is blocked using deep packet inspection and the likes. I use shadowsocks to access internet there but it requires additional server setup.

SSH Tunneling works with zero config so I use it as well, and rsp solves one of my major gripes with ssh tunnelling.

Take a loot at sshuttle [0] for the next level of this. Support other protocols transparently without messing with SOCKS support.

[0] https://github.com/sshuttle/sshuttle

sshuttle is amazing. I worked for a company that used it as its main and only "VPN" for years.

+1 on this I’ve used shuttle as the primary way to access one of my customers network for a year now, it haven’t gotten around to set up proper VPN access for external people yet. But it’s been working surprisingly well. I need to rerun the command a couple of times a bad day but other then that it’s behaved quite good as an ad-hoc VPN. You can also use a jump-host, I do this since I can only access ssh on one of the servers from the IP of one of my servers.

The only thing I could wish for would be an auto reconnect feature, been thinking about wrapping it in a shell loop to handle it but it requires sudo so I’ve been putting it of, I don’t like having a long running script with sudo.

How is this different to https://github.com/darkk/redsocks/ ? I am just asking because I have been using redsocks for a very long time and I am always open to improve things.

Redsocks is a tcp proxy and is layered on socks. Sshuttle is an IP proxy and is layered on ssh. They solve similar problems. Generally speaking sshuttle should perform better and work with non-tcp applications, for example udp, especially if you have multiple applications using it at the same time.

Yes, we used this for several years as a poor man's intranet as well as a vpn. Not great for windows users as I understand it but works great for linux and mac users. Easy to test if you have access to some ssh server.

Hi smw, I've added a section that mentions the sshuttle to the original article. Thanks for your suggestion!!!

While perhaps less portable (and a bit more proprietary) it's worth checking out the new project of Avery Pennarun, better known as apenwarr -- the creator of sshuttle -- which has made the rounds here, Tailscale [0].

I've switched all my devices to it and frankly, it's wonderful. Private VPN, tunnel, VLAN, auth, sharing, all-in-one.

0: https://tailscale.com

Can I ask why you chose Tailscale over the alternatives? I've been looking at that spectrum of products and Tailscale does look nice.

One thing I can't get over is that it's pricing is "per user". That makes sense our user-oriented VPNs (dev workstations, home machines, and phones), but doesn't feel like it's oriented towards server use. My end use case is probably 80% securing server connectivity across sites.

Alternatives I've looked at include: ZeroTier, Slack Nebula, OpenVPN Cloud, and PriTunl.

ATM, I'm using ZeroTier for my home use and quite happy with it. I had hopes of being able to deploy something for work as well, which is why I leaned towards ZeroTier. For home use, TailScale's pricing wasn't a concern.

I’m considering something like this as well and my two current contenders are Nebula and ZeroTier. Would you say there’s and specific reason you singled out ZeroTier and is there anything you miss?

Nebula looks great for server-to-server connections for things like administrative management/control or the like. The fact that it's fully free means we wouldn't have to deal with licensing, unfortunately I can find it hard to get approval for licensing and purchasing.

I like ZeroTier because: It was dead simple to set up, the web management UI is good, the "public" networks seemed like they might be nice for CI/CD servers to connect into some infrastructure to deploy (though that didn't work out because CI/CD docker didn't have flags allowing ZeroTier to work, but I did just get SSH over userspace ZeroTier working). It's super easy to set up in all the environments I've looked at.

For one, ZeroTier does not necessarily want to authenticate you via a public OAuth provider. I'd rather not rely on it where not strictly necessary.

Then, ZeroTier does have a free tier, with open-source clients for every platform I care about. Yes, they are limited to version 1.x, but maintained nevertheless.

I am currently using Tailscale just within the free "Solo" pricing tier, so cost has not been an issue. Compared to alternatives, it's as close to fire-and-forget as I think you can get. Everything just works, and they've been working hard to keep it that way. The sort of software I like.

If you're a larger business, the pricing seems... fine? I don't know what their enterprising pricing is like, but the next-highest tier appears to cap at $10k/yr, which seems like a trivial cost to anyone operating at that scale.

Consider also that you don't need to run tailscale on every host -- though you could. Depending on your architecture, a single gateway instance in each VPC could be sufficient. In other words, I could probably scale the two non-enterprise tiers very, very far, at least until you had close to ~500 employees, in which case you probably want to strike a deal for custom support and SAML SSO anyways.

Alternatively, at $LAST_JOB we just ran our own OpenVPN servers and in-house beyondcorp authwalls. There was honestly not a lot of complexity in there. We had <3000 employees.

> That makes sense our user-oriented VPNs (dev workstations, home machines, and phones), but doesn't feel like it's oriented towards server use.

Just to be clear, you don't pay per-server. Just for logins.

> My end use case is probably 80% securing server connectivity across sites.

100% is not by any means an unachievable goal in, say, a quarter or two. Any reason why you don't want that 20% tail?

To clarify, I meant 80% of my use is server to server communications, 20% is user to workstation or server use.

Ah that makes a lot more sense. I run tailscale on a subset of my servers, but I use tailscale to talk to all of them.

For example, I can use my desktop at home as an exit node for my phone, so I can configure my network hardware (router, switch, AP) from my phone, from anywhere.

I tied to use sshuttle on macOS client. But somehow the https://www.whatismyip.com/ still shows my client IP instead of the ssh-server's IP. On the other hand, https://www.whatismyip.com.tw/tw/ shows the ssh-server's IP correctly. I wonder if anyone knows the reason...

Sorta related;

I often travel for work and was having an annoying time with Verizon’s hotspot throttling. 4G LTE should be able to run at ~5Mbps but devices connected to the hotspot get throttled down to an insufferable 400Kbps. This was super annoying and felt like an arbitrary action on Verizon’s part. I found out I could run a SOCKs proxy on my iPhone via a Pythonista script and tunnel hotspot connections through it to fool Verizon’s throttling systems. Worked great, even though the UX of launching a Pythonista script as a service and pointing clients to it was slightly clunky.

I do love me some Pythonista, but it never occurred to me to run an SSH host in it. I wonder if this would be any easier today using iSH instead.

I'll have to look into it! Now that I've got the 5G-UW plan from Verizon, I wonder if I could actually get >1Gbps to my laptop via wired hotspot.

Some mobile providers have two distinct APNs (access point names) - one for mobile data and one for tethering. In this case, you can rename the tethering one to the mobile data one.

I think I noticed something similar back in the day with Straight Talk (ATT). Speeds when tethered were much worse than on my phone with most of my modern laptops, but when I tried IE on an old XP install I had I saw significantly faster speeds than my phone/plan were supposed to be capable of.

Apparently they detect hotspot usage using the IP TTL value. If its 1 lower than expected, then it knows there was an extra hop. If you know how, you should be able to just bump up the TTL value sent from your device.

I use this with .bashrc aliases to quickly fire up the proxy:

  # Temporary Firefox session commands.
  alias socks-proxy="ssh -D 50000 -N me@some-server"
  alias firefox-proxy="firefox -P \"Proxy\""

Then I open two terminals and run `socks-proxy` in the first and `firefox-proxy` in the second. The Firefox profile "Proxy" has to be configured to use the localhost:50000 SOCKS proxy on first run.

While we're here, I also have an alias that generates a random profile directory for Firefox in a temporary directory, to use websites that detect ad blockers:

  alias firefox-throwaway="firefox -no-remote -profile $(mktemp -d)"

Different use-cases of course, but I quite like foxyproxy for rapidly switching from one proxy to another, or based on patterns.

I don't use openvpn anymore for work, and instead make do with foxyproxy+ssh. I should probably try to hop from a server with wireguard to help with roaming.

I used to use this feature all the time when I was in school. When I lived in the dorms they limited our external connection speed to 8Mbps but all of the department servers I had access to weren't limited (by anything other than the school's connection speed and dated network infrastructure), so if I routed my connection through one of them I could get speeds closer to 100Mbps. It also came in handy if I needed to use sketchy wifi at a motel or something.

Haha, oh yeah, I did that too. I forgot about using it to circumvent throttling. Back then I felt like superman when I got 100mbps Internet on my PC!

I did this, too, but to circumvent the school's internet filters. I routed it through my home internet, which was excruciatingly slow but better than nothing.

This works in reverse as well, which is useful to use your laptop as a proxy. Suppose you can SSH to a server that can't connect to something your laptop can, like a git server accessible only over VPN.

  ssh -D8081 localhost
  ssh -R8081:localhost:8081 some.remote.server
  http_proxy=socks5h://localhost:8081 git clone git@git.corp:foo/bar.git

Actual VPN over SSH? The TUN device:

-w local_tun[:remote_tun]

Requests tunnel device forwarding with the specified tun(4) devices between the client (local_tun) and the server (remote_tun).

The devices may be specified by numerical ID or the keyword “any”, which uses the next available tunnel device. If remote_tun is not specified, it defaults to “any”. See also the Tunnel and TunnelDevice directives in ssh_config(5).

If the Tunnel directive is unset, it will be set to the default tunnel mode, which is “point-to-point”. If a different Tunnel forwarding mode it desired, then it should be specified before -w

Example of this at [0].

[0] https://rkeene.org/viewer/tmp/ssh-ip-tunnel.txt.htm

This will suffer from TCP-over-TCP performance issues. Projects like sshuttle, mentioned above, get around this.

Combined with stunnel, https://www.stunnel.org/, you can use ssh as a "VPN" that looks like regular HTTPS TLS traffic over port 443! Great for getting around being stuck behind a firewall that locks down common ports and sniffs ssh connections.

On machines where I have a web server already running I like to expose an HTTPS "CONNECT" proxy which only allows connections to localhost:22 for this purpose. On the client side you can use "proxytunnel" as the ProxyCommand.

Is "VPN" what everyone calls "proxies" these days?

By the way, SSH has real VPN option (-w option)

Can it transport UDP packets?

Yes. It initializes tun interface on both ends, so you can transport any IP packets.

No, just the uninformed do so.

We've lost that one along with "literally". Unless we find a way to kill all the "privacy VPN" services advertised on the internet, we may as well just name the real VPN something else.

While it is inaccurate to call it VPN, it's also not descriptive to call it a proxy. It's more than that; it's really a proxy split in half with a private tunnel between the two halves.

So really neither term is good. I don't know of a better one, though.

This title was pretty confusing. I assumed it meant running a VPN connection over SSH (for instance, by tunnelling).

This was a common technique in my halls of residence as an undergraduate -- tunnel OpenVPN over SSH, plug in an secondary Ethernet card to your computer, then bridge the virtual adapter and the new Ethernet adapter. You could then plug in your Xbox/PlayStation into the ethernet adapter and bypass the rather strict filtering that was in place.

One very useful program on Linux is "tsocks" which allows you to use many command line programs transparently with a SOCKS proxy. It works using LD_PRELOAD to intercept libc network functions. It hasn't had much development work recently, however.

I don't have much experience with stunnel, but I have used sshuttle and SSH SOCKS proxy/ tunneling, SSHFS and other tricks quite a bit also for work. I have done SSH over Tor as a hidden service but I haven't played with obfsproxy just yet. Some people made tunnels (including SSH) over DNS, which can be handy as well. This is probably just enough to check HN, read email and SSH/ Mosh to somewhere to fix something when travelling or so. If you understand SSH (e.g. by reading the book by Michael W Lucas: SSH Mastery: https://mwl.io/nonfiction/tools#ssh) you will probably come up with your own tricks. I have written some of my tricks down in this OrgPage: https://www.orgpad.com/s/UHUor4 there are screenshots for Linux and Windows for some things related to SSHFS, SOCKS Proxy and more. From time to time, I update it to reflect new tricks.

One of the newest tricks I haven't written down just yet is tunnelling a TCP port of a different machine than what you connect to over SSH. This is good for connecting to that Windows XP machine you have no control over (since it probably controls some industrial machine) but that you have to provide access to to certain people e.g. for maintenance of the industrial machine. This works reliably for e.g. tunnelling VNC, RDP and even Samba/CIFS for the occasional file transfer (e.g. a new executable file of some industrial control software). If you have no means to do a proper VPN, SSH is installed pretty much everywhere on current OSes (even current Windows 10 ships with an SSH client).

I used to do this around 2004 to circumvent school and later university networks and the like. I used to do it whenever using an untrusted network like free wifi too. Things were simpler back then. Later I used a thing called http tunnel to break out of more evil networks that blocked all ssh traffic. Nowadays with deep packet inspection and ip whitelisting on those kinds of network even that is very difficult.

SSH/SOCKS over an iodine DNS tunnel on captive WIFI was always fun, if not that usable.

If you control both the local and remote side fully and know how to do networking: OpenSSH not only offers L3, but also L2 network tunnels. Before Wireguard, I had been using this feature for years for ad-hoc and not so ad-hoc tunneling. Can be wired up easily using ip-route and ip-rule on Linux. Use systemd units for connecting and reconnecting.

Aah, the ghetto VPN.

I have an autossh session on my workstation along with the ControlPath trick mentioned elsewhere that meets most of my networking needs when working from home. It's not really practical for high bandwidth applications, but for text based interactoins, ssh -D is wonderful.

I wrote an article about using SSH through websocket:

https://rumpelsepp.org/blog/ssh-through-websocket/

Further, with socat IP traffic can easily be tunneled (trough websocket :D):

https://rumpelsepp.org/blog/vpn-over-ssh/

I spent a lot of time in China and ExpressVPN was the norm for the Americans I hung out with. Sometimes though, Express would be down, and I'd spin up a Socks proxy pointing at a Digital Ocean droplet and it worked like a charm. The downside is that only web browsing over Firefox (or the browser configured with the Socks port) will use that proxy, unlike OS or non-browsing internet traffic.

So what if you created a VM which runs the proxy and then you run your host traffic through the gateway on the vm? Could that work?

There are solutions which allow to redirect transit TCP connections into SOCKS proxy transparently: https://github.com/cybozu-go/transocks

So, yes, VM/router solution will work for TCP connections.

Also there are wrappers which can intercept connect calls to libc from applications on Linux (which is useful if you need to redirect only one application which doesn't support SOCKS proxies): https://linux.die.net/man/8/tsocks

> unlike OS or non-browsing internet traffic.

Not nearly everything else, but alot of things use PROXY/HTTP_PROXY for HTTP things which works here too.

You can use mellow[1] to turn your socks proxy to transparent global proxy. Although this software is not maintained anymore, it still works.

[1]: https://github.com/mellow-io/mellow

I use a lot ssh tunnels of all kinds, they are very for mapping local/remote ports to remote/local servers: database sockets, http ports etc. Combined with proxy.pac you can seamlessly access different remote resources from the browser.

I wrote a small gui application for mac/linux to start tunnels and control them via notification area (menu in macos) icon.

If you want to do it on android https://github.com/madacol/knowledge/blob/master/Ssh%20poor-...

I've found ProxyJump to be a bit simpler than ProxyCommand -- it might be a newer feature. For example, my SSH config has two entries to connect to my grandfather's laptop through a Raspberry Pi that acts as the jump host:

  Host papa
  HostName 192.160.0.1 <-- LAN IP address
  User alex
  ProxyJump papa_pi

  Host papa_pi
  HostName 1.2.3.4 <-- public IP address
  User pi

Socks Proxies are fun. I used to use SSH on a non-standard open port on GoGo inflight WiFi on American Airlines to get free inflight WiFi on continental flights. Don't know if it still works on the newer systems, but was fun and convenient.

Sidenote: does anyone know how Telegram bypasses the WiFi redirect blocks? Was considering writing a Reddit or HTTP-over-Telegram bot but it seems easier to just figure out how they are avoiding blocks.

just use tsocks. it ld_preloads over the socket functions and has a configurable table for what to redirect through the socks server (in my case ssh). it makes any program socks aware.

net result: with tsocks bash or tsocks command, you can control which processes or shells can see the vpn remotes.

tsocks + ssh -D has been my preferred vpn solution for many, many years.

I tried to use similar LD_PRELOAD solutions but I always ended up finding some programs that don't go through libc, so they bypassed the VPN.

badvpn-tun2socks works with everything I wanted, but it's not as easy to control the processes that go through the VPN. It should be possible with network namespaces or tagging packets with cgroups but it's a pain to get it all set up.

i'm curious which programs bypass libc for socket stuff. making direct syscalls is exceedingly rare in my experience.

I don't remember exactly, but I believe I was related to Java, probably trying to debug some Java app with IntelliJ. I'm not 100% sure if the problem was due to some intrinsic LD_PRELOAD/libc limitation or something related to the tools I tried, but trying to find what the problem was with something so gigantic seemed impossible so I went the tun2socks way which is more transparent for applications.

(Also Go programs are famous for bypassing libc.)

hmm. that's super weird. at the very least i would expect them to use the low level syscall wrappers in the libc. unless they're avoiding linking libc at all, it seems super weird to be making syscalls directly from application code.

i mean, what are they doing? implementing the syscall gate by hand. why would anybody do that?

anyway the reason i bring it up is that i've been toying with the idea of lightweight ld_preload "containers" which patch the relevant syscalls to mount additional filesystem and network resources in a safe and extremely high performance manner by bypassing the kernel completely.

ahh. i see now.. portability... i bet it would be possible to bypass/patch sysenter to achieve the same aim even if they're implementing their own syscall gates.

Maybe because the binary was statically linked. Go binaries are.

> it's not as easy to control the processes that go through the VPN. It should be possible with network namespaces or tagging packets with cgroups but it's a pain to get it all set up.

OpenBSD routing domains make this really easy.

I did this for years until recently. Now I use wireguard instead. Now I don't need a giant ssh config with stuff like:

  Host myhouse.net
        LocalForward 5902 foo:5900
        LocalForward 3393 bar:3389
        DynamicForward 8083
  Host baz
        ProxyCommand=nc -X 5 -x localhost:8083 %h %p

It depends on your use case. I use a SOCKS proxy with some applications and not others to do a little bit towards hiding my residential IP. Application-level proxy support is much easier with SOCKS because it's commonly supported. I also have a Firefox extension that can be used to route some URLs via the proxy and others not, because the browser extension API for setting per-request proxies supports SOCKS proxies.

With Wireguard I'd have to create routes to route some some destination IPs over the wg interface, which not only needs management to keep up-to-date with DNS changes but also does not have any way to behave differently for routed applications vs unrouted applications.

> I also have a Firefox extension that can be used to route some URLs via the proxy and others not

FoxyProxy? I love that one.

https://addons.mozilla.org/en-US/firefox/addon/foxyproxy-sta...

No, I wrote my own. I avoid third-party extensions because the risk is too high.

Is there any guides or libraries to help you writing your own proxy switch extension? I have considered writing my own for the same reason, but I don’t know where to start. I have experience in writing simple WebExtension, I just don’t know how to write a socks proxy in JS.

Thanks in advance.

https://gist.github.com/Arnavion/b8d3fa6e6e3efa57fd4c45ada82...

Thank you, it looks so easy!

Usually when I read something like this I am worried about the false sense of security that SOCKS proxy users give themselves.

SOCKS is a very outdated protocol, that's why I wrote an in-depth article about it. The TLDR is kinda like this:

- don't use socks proxy passwords, they're plaintext or work only with pre-shadow BSD 0.x.

- IPv6 doesn't work. TOR socks proxy supports it but no Browser does.

- DNS isn't tunneled (only enforced DNS over TLS is due to http upgrade nature), and TTL of 0 (which is pretty much always everywhere) leads to the ISP / spy agencies being able to correlate your internet connection regardless.

- Most SOCKS implementation if not all that I've seen don't support "bind" correctly and neither work with UDP connection handshakes.

- therefore SOCKS in practice is TCP only, http upgrades will always circumvent a socks proxy because it cannot handle those connections.

- instead of a socks proxy I'd recommend to use a simple https proxy that uses a CONNECT header which has the identical features as a socks proxy in practice, but could be implemented in a way that it is compatible with http3/quic upgrades and websocket upgrades.

Source: The "RFCs" (which are two specs of an already proprietary implemented protocol at the time) and my Implementer's Guide to SOCKS that I posted on HN a while ago and didn't get any traction [1]

[1] https://cookie.engineer/weblog/articles/implementers-guide-t...

> IPv6 doesn't work. TOR socks proxy supports it but no Browser does.

Firefox 87.0 on GNU/Linux with an OpenSSH 7.9 SOCKS proxy (-D) does make IPv6 connections from the correct destination address. Is there any other sense in which it does not work? One thing I do is bind on ::1 locally instead of 127.0.0.1

Cool, never thought ssh can start a socks server!

Yup, it's an older feature, but it checks out. (:

I do this all the time and run one browser with the socks proxy and another instance non-proxied.

What can you do if you cannot change the proxy settings? Disabled by an administrator.

If I set this up with Firefox, would DNS look-up also use SOCKS?

Notice the fourth yellow circle in the image.

Aha! Ty!

That setting is misleading. It just toggles whether DNS resolution happens on client side (the machine the browser is running on) or on SOCKS server side. As the SSH client acts as SOCKS server and as the SSH client usually runs on the same machine as the browser, this setting doesn't make any difference when using SSH as SOCKS proxy!

When using SOCKS over SSH and running the SSH client on your local machine, DNS resolution will always happen on you local machine as well (except when using DoH of course)!

It might seem misleading if you're using ssh as a SOCKS proxy, but there are many other SOCKS proxies for which the setting makes sense. For example, if you're using tor as a SOCKS proxy, then that setting is the only way to have the browser resolve .onion addresses.

TCP over TCP is not the best idea, but it works in a pinch.

Edit: see below

SSH/SOCKS tunnel is not TCP over TCP.

Whoops, you're right. I didn't know that SOCKS only forwards data, not packets.

Use this all the time. It's very useful.

Reply on news.ycombinator.com