What's wrong with exclude and exclude-from? Using git rm without file manager integration - can be a little thorny.