This is because C: has additional file system filters for system integrity that are not present on other drives; and also because C: has 8.3 name compatibility enabled by default, but additional NTFS partitions have that disabled. (so if your filenames are longer than the 8.3 format, Windows has to write twice the number of directory entries)
Even excluding directories (like D:\MyProject) from defender isn't a full solution, the fs filter will still use the file handle to do lookup of the file path to see if it is in an excluded directory (at least on win10), but excluding an entire drive (like D:\) does solve this.
I have found that adding git.exe to the process exclusion list makes the biggest difference here, although git is still slow when dealing with lots of files, it goes from unbearable to bearable.