Hacker News Reader

Running over 80 and 443 using an arbitrary protocol often doesn't even work with next gen firewalls because they don't recognize the protocol as HTTP.

Curious how that would work on 443, given that the whole point of SSL is that you can't recognize the payload as...well, anything.

Payload aside, SSL is identifiable as SSL. The host name is transmitted in clear text (SNI) as well as the protocol version and cipher negotiation.

Just because the traffic is encrypted, it doesn't mean the connection type isn't identifiable.

Right, but presumably any arbitrary protocol that plans to tunnel over port 443 would use a standard SSL negotiation, standard HTTPS connection, and then send the application-level protocol as part of the encrypted payload.

My very first job (back in 2000) actually did exactly this - we tunneled arbitrary TCP/IP traffic over HTTPS. The startup died over various management & investor relations fuck-ups, but the product was working - you could do things like make a SourceSafe connection across a firewall without exposing the server to the Internet. There was some concern about whether firewalls would kill the connection if it was open for too long, but it turned out that none of them actually did this (which unfortunately we didn't figure out until we were almost out of money and had wasted several months handling this case; see again re: management fuck-ups).

That's a different point though as you're now effectively using HTTP as your transport layer rather than using your proprietary protocol "naked" on port 443. So whether you send your protocol in a proprietary binary format over HTTP or via JSON API's, it's still just HTTP over TLS. You're still sending HTTP headers, just with our own proprietary content as the body. The real problem is if you want persistent types of connections. As websockets have taught us: a great many equipment that supports HTTP don't support websockets and thus why even websites are still often stuck with RESTful APIs even when that doesn't always fit their problem quite right (that situation is improving however).

If one is using HTTP to transport your protocol (and the aforementioned websockets point isn't an issue) then your protocol should work over port 80 as well as 443. Barring any corporate / nationwide web filtering; but as others have already pointed out, that could also be an issue with 443 even with advantage of TLS.

By the way, was it Socks2HTTP you worked on? I remember that project back around the time you described and was quite fascinated by it.

> That's a different point though as you're now effectively using HTTP as your transport layer rather than using your proprietary protocol "naked" on port 443.

HTTPS is built on top of TLS, not the other way around. You can't (passively) tell if a 443 TLS connection is HTTPS or a proprietary stream. You can take a guess based on statistics (which is what the Chinese firewall does to detect tunneling) but that's about it.

I'm aware HTTP is built on top of TLS in HTTPS. I think what you're discussing now is a little different from the HTTPS as a transport vs bespoke protocols running "naked" over TCP/IP (unless I've badly misunderstood any of the previous posts?).

Warning: nonsensical brain dump follows:

However to address your point, you might be able to use SNI (which is sent from the client before the TLS connection is encrypted) to make some assumptions about the content. Granted this would be more in the realm of web filtering where you'd blacklist suspect domains or - in extreme cases - banned terms within hostnames. I wouldn't be surprised if SNI is one of the "statistics" the Chinese firewall uses (I'm not familiar with the implementation details of the Great Firewall of China")

A middlebox has no way[1] to tell what protocol is inside of a TLS connection unless it terminates TLS. It could be HTTP, it could be telnet, it could be whatever. So no, you're not effectively using HTTP as your transport.

1. Almost no way. The great firewall of china has traffic models of what payload sizes should look like for upload and download traffic of standard HTTP over TLS. If you start to tunnel TCP/IP over it, the pattern changes enough (small payloads for TCP ACKs, etc) that they will inject a TCP RST into the stream to screw up the connection. It's impressive and super frustrating.

> A middlebox has no way to tell what protocol is inside of a TLS connection unless it terminates TLS. It could be HTTP, it could be telnet, it could be whatever. So no, you're not effectively using HTTP as your transport.

While you're right that you could just use TLS without HTTP (as a great many services already do), the comment I was replying to was talking about running over HTTPS (he specifically stated HTTPS), ie TLS + HTTP. Which is effectively just using HTTP as your transport.

Like everything in IT, there's multiple ways one can approach this problem and it's probably fair to say that using TLS without wrapping your data inside a HTTP body makes greater sense if you're writing your own protocol from scratch. But on this occasion the post I was replying to - and many of the posts that preceded it - did make frequent references to HTTP and HTTPS.

> The great firewall of china has traffic models of what payload sizes should look like for upload and download traffic of standard HTTP over TLS. If you start to tunnel TCP/IP over it, the pattern changes enough (small payloads for TCP ACKs, etc) that they will inject a TCP RST into the stream to screw up the connection. It's impressive and super frustrating.

Ouch. Impressive though. Could one change the payload sizes of the TLS connection to make it look more like HTTP traffic? Shouldn't be that hard to do as most of the time you'd probably just need to add junk to the end of the server replies (assuming you have a client / server relationship with your TLS protocol). You'd probably need to make the TLS connection RESTful as well - to further mimic HTTPS and the limited connection times. Though before long you've just reinvented HTTP....

By the way, how well does websockets work over TLS? Do they throw up false positives on the Chinese firewall?

I've used sshuttle once, which is an IP-over-SSH program. Great stuff but it's kind of sad how much extra work firewalls create.

> Great stuff but it's kind of sad how much extra work firewalls create.

That's a weird statement to make given the point of firewalls is to limit ones access to a particular resource. If you're the systems / network administrator then the extra work is part of the job securing your infrastructure. If you're not an administrator then you're bypassing the security measures put in place by your administrators - which may well be in breach of your employment contract (as they often outline IT policies). Worse yet, if you're not even hired by whatever company nor individual who owns that infrastructure, then what you are doing is illegal.

Some companies MitM the traffic (installing their cert on all computers) so there's that.

Companies have the privilege of blocking traffic on their network.

If they agree on letting employees use a service they'll let the protocol go through the MitM. Not only that, they'll let it go through the firewall on its native port without the need of tunneling it into https.

But if a company blocks a service, employees should not circumvent the block. That would be risky.

That view may be ethically valid, but anecdotally, normal employees are more creative in their pursuit to get the job done than most hackers.

I know and you can just use your phone nowadays. But be careful at doing what you shouldn't contractually do. If your boss is looking for a way to fire you, you're helping him even in those countries with strong protections for the employees.

Depends if the network your on is state level and/or corporate ownership of device level MitM or not.

And as my view in that post foretells, I believe Twitter is already a protocol, we just call it the Twitter API. It's not an open protocol, of course, and its domain model is fairly anemic [1], especially compared to that of Facebook [2][3][4]. But in my opinion, this is what the author is trying to say: Twitter could have been a place where people interact with structured resources created and consumed by other apps.

It's a possibility, especially since Facebook became that place instead. Ultimately, Twitter didn't because they wanted to focus on driving more traffic through their first-party app (presumably as a captive client on which they can eventually display ads), and because they focused on cultivating a community (like Medium, Tumblr, LiveJournal) rather than a resource ecosystem.

[1] https://dev.twitter.com/rest/public [2] https://developers.facebook.com/docs/graph-api/reference [3] https://developers.facebook.com/docs/sharing/opengraph/using... [4] https://developers.facebook.com/docs/sharing/opengraph/objec...

> "and they probably get blocked on a middlebox except if you use port 80 or 443 and fake being HTTP anyway"

Curiously, games using UDP still work fine. Stop justifying bad protocols with the proxy straw man.