I also keep my whole homedir in git, and it's great. My approach isn't to .gitignore, because I want to ignore almost everything -- I just add and commit things when I want to track them. One of my many git aliases[1] is 'git sn' for showing status without untracked files; this helps when dealing with my homedir:
status --short --branch --untracked=no
[1] https://github.com/cespare/dotfiles/blob/master/.gitconfig#L...