Do you have your projects cloned in the remote host?
I ask because I was excited about this prospect only to learn that I could only use Remote: Containers if codebase and container are on the same machine as the project workspace is just volume-mounted in the container.