No, you don't need to restart your machine after you install CUDA.
Also you might not need to restart after you install the drivers, this is not Windows. (But there might be some rmmod/modprobe needed)
> If your deep learning machine is not your primary work desktop, it helps to be able to access it remotely
Yes, use ssh.