Self signed is more, than enough for the same purpose certbot exists: to have encrypted communication.
LE and other certificate authorities* at least prove that your data wasn't tampered with in transit, or wasn't decrypted, read and reencrypted without any way for you to find out.
* as long as you trust those authorities
I do not enough to solve that problem really.
So I'm going to disagree. No, there is no security benefit to encryption without authentication.
There certainly are real world situations where the chance of active eavesdropping (which requires a targeted attack, and is risky because it can be detected) is low, so you accept that level of risk, but the ability for someone to just grab all the traffic of a large group (that just happens to include you) for a prolonged period of time undetected is much higher.
Encryption prevents sniffing, Encryption + trust prevents sniffing and MITM. Providing sniffing protection is an upgrade.
As an example, say HTTP would always upgrade to HTTPS with a server provided public key. That would be more secure than the HTTP we have today, even though it wouldn't prevent MITM.
Of course it's worse when the user thinks the connection is encrypted when he actually has no idea who he's talking to.
What kind of attack do you have in mind where someone can sniff on the data but not tamper with it?
And how should a browser explain this situation to the user? No, the mindset that an unverified certificate is better than no encryption is very dangerous.
If a website previously using a self-signed certificate switches to plain HTTP - how will that help me verify the identity of the server the next time I visit?
By removing the self-signed certificate, not only am I still unable to verify the identity of the server, but now my traffic is in plaintext for anyone on the local network to trivially intercept (in addition to whatever stranger I'm sending it to on the other end).
I understand your sentiment, and I know the slippery slope that you are referring to when you say that it's a dangerous mindset to be okay with unverified certificates. Unencrypted communication however, is not a solution to that problem.
Could you point out who you are responding to who said that unencrypted communication is a solution to the problem? This strikes me as a straw man argument.
If they are able to trivially intercept your network traffic they are probably also able to modify it (=> hijack untrusted HTTPS) or what scenario am I missing here?
Of course unencrypted communication isn't a solution if your goal is to have secure communication. But so isn't untrusted communication.
Either it's secure or not. You can't have something in-between. The browser would have to display an icon that says "This connection is secure but actually we don't really know so maybe it isn't". What are you supposed to make of such information?
The NSA for example is known to just suck up all the traffic it can get and put it in a pile for later analysis.
Maybe your mention of "Make a bomb in chem class tomorrow" was just a joke to a close friend about how much you hate school, and maybe an analyst will realise that and move on when they see it, but civil liberties advocates think it'd be better if that analyst couldn't type "bomb" into an NSA search engine and see every mention of the word by anybody in your city in the last six weeks. I agree.
Americans tried just telling the NSA not to collect this data, but the whole point of spooks is to do this stuff, short of terminating the agency they were always going to collect this data, it's in their nature. So the practical way forward is to encrypt everything.
Any TLS connection can't be snooped. Only the participants get to see the data. The NSA isn't going to live MITM every single TLS connection so even with self-signed certificates the effect is you prevent mass surveillance.
A targeted attack will MITM you, no doubt, and so that is the reason to insist on certificates, but it's wrong to insist as you do that there's no benefit without them.
Ok, that wasn't really my intention. I was stating that a false sense of security is worse than having (knowingly!) no security at all.
So yes I agree, you're generally better off even with untrusted encryption but that doesn't help in practical terms with our current situation of HTTPS in web-browsers. Maybe it would have been better if web-browsers would have just silently accepted self-signed certificates while still showing the big red warning about an insecure connection. I guess that will be solved with QUIC/HTTP3.
Agreed. If you know that you are insecure you're less likely to pass sensitive information over the connection.
IMO the culprit is browser behavior. For instance, when visiting unencrypted HTTP sites in Chrome you may or may not notice an unobtrusive, greyed out "Not Secure" label in the URL bar. Visit your own self-signed certificate dev site though, and Chrome will give you an error wall with nothing to click, and you have to type "thisisunsafe" to pass (the page does not tell you that typing "thisisunsafe" will get you through).
Perhaps the reasoning is that if a site is served unencrypted it shouldn't be serving sensitive information, whereas an invalid certificate is an easy indicator of something amiss... but wow, talk about obtuse.
Your concern is definitely valid though, and I'm concerned about it too.
If the site doesn't seem to require HTTPS but you've gone there with HTTPS and there's no trustworthy certificate then the browser gives you a different interstitial which has a button labelled Advanced which reveals a link "Proceed to ... (unsafe)" that will switch off further interstitials for this site but retain the "Not Secure" labelling.
The HTTPS site (once you reach it) gets access to all modern features, an HTTP site, even if you ignore all the warnings, does not. As an example that's particularly unsubtle, calls to all the WebAuthn APIs just give back an error as if the user has thumped "Cancel".
Edited to add: Also the grey "Not secure" is changed to red if you seem to interact with a form, because that's probably a terrible idea on an HTTP site. Eventually I expect it will just always be red (the change to notice form interactions was in 2018 and this is part of a planned gradual shift by Chrome and other browser vendors).
It took Letsencrypt to make HTTPS accessible to the majority of the web because there was no cheap way before, because self-signed certs were punished by browsers while unencrypted connections were fine. We could have been full on moving from an encrypted (self-signed) web to a trusted (CA) web by now instead of moving from a plain-text to a trusted web.
Also, self-signed certs still prevent a MITM if you ever connected to the site before, similar to the trust-on-first-connection behavior of SSH. Given the widespread deployment and trust of SSH I'm suprised this people act so different with HTTPS.
Passive collection, where you search through the accumulated data after the fact.
In what real-world situation is that an upgrade?
The browsers agree with you. Blame Netscape (a corporation which no longer exists) in the mid-1990s (when people still thought Bill Clinton was cool and that the Web might be a fad) for this UI design mistake. Or maybe even Sir Tim himself since his "Web" has no security whatsoever.
But, merely acknowledging that the present design is wrong doesn't fix it, and we can't rewrite history. So the shift has been gradual, but it's real.
In this current Firefox for example an HTTPS site which presents a self-signed cert gets a warning interstitial, and then a cautionary UI marker but once I tell Firefox I trust this certificate it functions normally.
On the other hand an HTTP site gets a red warning UI and some feature just don't work, and there is no way for the site to add those things, they're just outright prohibited with HTTP. If I try to fill out a form the browser reminds me it's insecure, and again there's no way to switch that off because it's true.
Long term the goal is to deprecate HTTP entirely. You will have to use any remaining HTTP sites through a reverse proxy (or an old browser), perhaps somebody benevolent will operate a public proxy for those rare public sites that refused or are unable to upgrade to HTTPS. I'd guess we're maybe 5-10 years from that.
This is a very recent development. Until last year HTTP had no special behavior while self-signed HTTPS had a big "You WILL get hacked and all your creditcards stolen if you continue, do not continue unless you have a PhD in Computer Security" warning. Still has.
Chrome shows a grey "not secure" on HTTP and a similar warning on self-signed.
In terms of how secure HTTP vs self-signed is, the behavior should be reversed.