>If an active attack is carried out, I'm reasonably sure people would not notice.
If the attack happened after you have started communicating wouldn't you get a notification that the key of those you're communicating with has changed? An attack would only be invisible if it happened from the first communication. But unless it's very targeted someone somewhere would compare the keys and notice.