Windows 10 IoT Core has been an option for Raspberry Pi for a while. It's a mostly headless (just the bare amount of UWP GUI support, no Win32), desktop-less SKU intended for embedded installs.
I couldn't think of anything more horrible than running Windows without a GUI. Why not use Linux at that point?
It's the same concept. For linux you'd use SSH which gets you to a shell for remote administration, for Windows you use powershell via the Enter-PSSession command.