If you have a lot of files or some large files it can slow things down a lot. The git server is something that's due for a revamp, but as it stands the best thing you can do for now is separate out your LaTeX content and other project assets into different repositories. (We usually see these kind of problems when people have a bunch of non-document stuff in their Overleaf project)