In WSL 2, Linux accessing Windows files is very slow, and Linux accessing Linux files is fast.
A couple of years back Fastmail’s webmail build process did a lot of theoretically unnecessary process spawning and file rereading (spawning Node.js several thousand times). I don’t have the exact figures on hand, but in WSL 1 with the files on C:, a full build would take about half an hour (files in WSL was not even worth considering, it took many hours). Under WSL 2 on C:, that full build would take an hour and a half. Under WSL with files under Linux, it was down to just a few minutes (partly because of faster I/O, partly because of faster process spawning). As I say, I don’t have the exact figures on hand, but that’s my recollection of the ballparks.
Like you, I steadily shifted just about all my stuff to the Linux side and reaped the benefits. After a while I even uninstalled Windows gVim altogether as I was always using Linux Vim in the terminal. (I start things in VcXsrv occasionally, but not often.)