It's likely a problem with Visual Studio being a 32-bit application and less to do with git.
This is why they referenced running git out of process to save a few hundred megabytes of memory.
By dropping libgit2, they freed up some memory. Obviously the overall memory usage of VS and `git` is probably _increased_ but VS teams are flogged until memory usage decreases because nobody will prioritize making it a 64 bit app. (Opposite, in fact: https://blogs.msdn.microsoft.com/ricom/2009/06/10/visual-stu...)
There's also the issue that Microsoft hired the maintainer of Git for Windows and lost a maintainer of libgit2.
It's not an ideal situation for libgit2, we certainly liked our relationship with the VS team, but we understand and sympathize with their decision.