I do something similar with vim, no gui. I use vim over ssh with tmux. Others I know use vs code with an ssh backend. These two make up the workflow of roughly 60000 Googlers. None of them need X forwarding.
Some people may benefit from X forwarding, but the vast majority of Linux users do not.