At least you are getting content from other channels. It seems like all my recommendations are to videos from the last few channels I have watched. Never anything I haven't heard of before. Even most of the channels I subscribe to never show up in my feed.
It's the old exploration vs exploitation tradeoff. Most recommendation algorithms are highly optimized for exploitation and not exploration.