Agreed. If you know that you are insecure you're less likely to pass sensitive information over the connection.
IMO the culprit is browser behavior. For instance, when visiting unencrypted HTTP sites in Chrome you may or may not notice an unobtrusive, greyed out "Not Secure" label in the URL bar. Visit your own self-signed certificate dev site though, and Chrome will give you an error wall with nothing to click, and you have to type "thisisunsafe" to pass (the page does not tell you that typing "thisisunsafe" will get you through).
Perhaps the reasoning is that if a site is served unencrypted it shouldn't be serving sensitive information, whereas an invalid certificate is an easy indicator of something amiss... but wow, talk about obtuse.
Your concern is definitely valid though, and I'm concerned about it too.