Kind of. It works very well if you own multiple computers that you want to be configured similarly (like a dotfiles repo on steroids).
It doesn't work so well for I borrowed my coworker's computer for 5 minutes and want to use my own Vim configuration.
However, Nix does also supply the "nix run" command, which allows you to use software without installing it permanently (but without NixOS' configuration support).
> Is there some way I can boot-to-git-configuration? So I could do this in RAM on someone else's hardware, like a network boot?
Yes. You could either build an image with a service that git clones your configuration and switches to it, or you could build an image containing a prebuilt snapshot of your configuration.
Either approach should be possible for basically any medium (USB drive, DVD, netboot, whatever). That said, for the build-and-switch setup I'd recommend something that can give at least /nix/store a writable partition (to avoid having to rebuild/redownload the whole world on every reboot).