Maybe you buy a lot of subscriptions and currency? I bet they avoid annoying their most profitable users. While it might have been coincidental, I never got ads until I unsubbed from the vast majority of streamers I follow. Same for other things like gifted subs. I used to receive a ton of gifted subs until I started lowering my sub count. They are obsessed with performance metrics. Big spenders have a very different experience.
Now, whenever I get an ad, I immediately close twitch and find something on YT/Mixer to watch. I'm training their algorithms to leave me alone.