Not having a #memes channel sounds like not having any trash bins in the house because you expect everyone to take their trash outside to the large bin / container. What actually happens is that the trash will litter the entire house.
How you deal with it is make it explicit that memes (or at least off-topic memes) are not allowed (and even excessive usage of on-topic meme is frowned upon), heavily enforce that rule and invite people who want to meme to join a meme server.
At least that's how it's done in a ~10K member server and it has worked well so far. Plus it avoids lowering the discussion level to meme-level.
vs
> make a meme channel and mute it
There's an order of magnitude difference in work required here