You think wrongly, they're definitely linked:
"Exercise: Resistance training increases testosterone levels acutely, however, in older men, that increase can be avoided by protein ingestion. Endurance training in men may lead to lower testosterone levels." [0]
Why on earth would you want to avoid it? That's like saying weight loss could be avoided by eating lots of cake.
"In people who have undergone testosterone deprivation therapy, testosterone increases beyond the castrate level have been shown to increase the rate of spread of an existing prostate cancer."
So there's probably myriad scenarios where one needs to balance testosterone levels with maintaining physical activity/strength training.