How do you know that? That's the whole point of SSL - knowing that you've traded private keys with the right party.
SSL for "encryption only" only works to defend against attackers that can listen to your network, but cannot write to it. So, sure, it defends against some passive collection system, and perhaps against some tools that are designed to just listen.
But, if browsers stopped displaying warnings, so that using a "bad" certificate worked just fine, then I'd bet the tools would just switch to allow cert injection and we'd all be worse off.