On linux a file that's open won't get deleted. It'll stay in memory. When you replace a file all it does is point the filename to the new file contents.

Any program open will keep running. Any newly started program will run the new binaries.

Of course this can give trouble with complex programs. Old executables seeing new config files, that sort of thing. But in general this doesn't seem to be a massive issue, and is fairly doable to work around by keeping critical files open all the time.