The issue in this case isn't caching, even if I'm not actually pulling any packages from our private nuget repo simply having that repo listed in the project's nuget.config file is enough to cause the issue.