You are nearly (and generally) correct about programs just stay running even when overwritten but of course programs generally consist of more than one file and might run more than once and if you update part of this and something reloads or whatever, you can end up in a world of hurt. That said, it is behaviour that I have relied on more often than I can count and so have you with your Firefox anecdote. I've done some remote Gentoo updates with systems in quite the broken state - it keeps IT interesting! I once had to get Puppet to install telnetd on some systems so I could repair sshd.