I've always found this plugin slow which is a shame. It's nice to visually see the changes made to a file.
EDIT: Here is the section on the read me about it: https://github.com/jisaacks/GitGutter#non-blocking-mode
I've done this with "big" files, as well as large folders and it works perfectly both locally and over a NFS share.
How many LoC are the files you editing?