You can avoid the downsides by being very informed. Sounds like you're still early in the journey.
I sympathize with being 3 sigma on the distribution... I'm four sigma out myself. My total T is around 1200ng/dl with most everything in my bloodwork related to hormones being a genetic freak. And, yes, I'm natural. Even with my extremely high T - probably twice yours - I struggle to gain any muscle at all and look like I've never lifted a weight in my life.
I'm still doing steroids next year because I'm tired of not seeing results for my insane effort.
If you're just growing in the stomach then you're doing something wrong. Check macros - should be overly high protein. Eat only 500 calories surplus per day. Your stomach will get larger just from food volume. Your stomach will always be large if you're on a bulk - it's supposed to be full all the time.
After you gain 20lb over about 20 weeks - reasses. As it is - it sounds like you haven't done really serious tracking. When I'm bulking/cutting - I don't eat anything unless every ingredient goes on a scale and I track everything down to the gram. One way to make less lifestyle less painful is to do meal prep and label all your meal tins with the macros. (Or just be obnoxious and measure everything out where each tin is the exact same - harder to do)
Get a dexascan a couple weeks into your bulk then a couple months in. You'll be able to tell if you're only gaining fat or if you're gaining lean body mass. Same process with cutting. Your eyes might not be well trained.