You can do the same with vscode btw. I also work on remote gpu workstations and vscode remote with ssh works just fine, especially if you use tmux, you can preserve all the history and what you were doing.
In my experience, vscode remote code editing is very buggy and resource hungry. Not very fitting if your Internet is unstable/slow and/or your target workstation is not that powerful (why is 8GB of RAM not enough?). I've had more success with Neovim either with distant.nvim[0] or directly on the Workstation with mosh[1]