one of the cool things about unix is (and perhaps windows can do this in the right modes, idk), the running copy of a program is a link to the code on the disk (a link is a reference to the file, without the file name). You can delete a running program from the disk and replace it with a new program, but the running copy will continue and not be aware that you've done that. You don't need to wait till the program finishes anything.
on an every day basis, this is what happens when you run software updates while you are still using your machine, even if your currently active programs get updated. you'll sometimes notice this in a program like Firefox, it will lose its ability to open new tabs; that's because they go out of their way to do that, they wouldn't have to if they wanted to avoid it, just fork existing processes.