If knowing you use a crate is too much, then running your own registry with a mirror of packages seems like all you could do.
Edit: A thought occurs to me. Cargo downloads metadata from crates.io but clones the package repo from GitHub/etc. So unless I'm missing something, downloading specific metadata instead of all metadata allows for crates.io to track your specific packages in addition to GitHub.
crates.io crates are tarballs stored in S3. The tarball downloads also go through a download-counting service, which is how you get download stats for all crates (it's not a tracker in the Google-is-watching-you sense, but just an integer increment in Postgres).
Use https://lib.rs/cargo-crev or source view on docs.rs to see the actual source code that has been uploaded by Cargo.