Not on all apps though. Office apps do not respect that shortcut convention and require clicking on the "Paste Without Formatting" button.
In Office, you can hit Ctrl after pasting, then "t" to select paste without formatting, which is nicer than switching from keyboard to mouse. The obnoxious thing is that, if the source text did not actually have formatting, then there is no menu that opens by Ctrl, and "t" is instead inserted verbatim. Wrecks havoc with my muscle memory, that I need to keep track of where I copied text from when pasting.
As much as that would be a helpful shortcut to remember, I know I won't. As a result, I just changed the default paste settings in Office.
That just removes formatting. zero-width characters are not formatting so they wouldn't be removed. They are as real as the letter A - you wouldn't want ctrl+shift+v to remove all A's from your text, would you?