Wait, you do not block HTTP referers by default? In this day and age where vast majority of websites are out to scoop any and all available information about you?
The fact that you don't send a Referer may ironically make it more easy to track you.
It’s information for fingerprinting, sure, but not sharing the page you’re visiting from for every link click is much more important for privacy IMO.
The best option I think is to spoof Referer to a random choice of known good values. This, of course, after disabling other, more reliable tracking vectors like JS and cookies that would just identify you as a Referer spoofer.
Anything you do online can be used to track you. The point is that by doing this, you plug an additional information leak about yourself.