Really? I work across multiple vscode projects (locally), some use dev-containers and others don't. I have never noticed any difference in experience across the two.
I have also used them remotely (ssh and using tailscale) and noticed a little lag, but nothing really distracting.