If you want to make changes to the upstream, I suggest adding an extra mirror in between the upstream and the repo you want to add the submodule in. That way, you can maintain your version without worrying about how your other code is affected. Only after it is tested do you then merge it into your other repo.