Does C# not require a multiple gigabyte download and admin rights?
FWIW I've gotten rust working with the GCC build tools on windows without admin rights, thought it's admittedly somewhat janky.
Not C# doesn’t need multiple gigabytes and admin rights. Dotnet tooling != Visual Studio
Is that needed because the compiler accesses some higher privileged APIs?
The MSVC-based compiler doesn't come with a linker, so you need the MSVC build tools to get it. One of the ways to get the build tools is to install Visual Studio, which is fairly large.