I bet Google picked Mercurial because it was easier for them to hack on and adapt to their infrastructure. A "libgit" is on lots of people's wish list, but the official developers seem pretty happy with the hodgepodge of binaries.
1. Google Code is built around HTTP, Git sucks at talking HTTP (it's an order of magnitude slower than hg). On the other hand, HTTP is hg's first-class protocol. So hg's integration was simpler here.
2. Windows still is a first-class platform for mercurial, still isn't for git, and windows users are an important part of Google Code's clientele.