I’ve heard this position multiple times, and yet every single benchmark I’ve seen repeated by teams of engineers in multiple contexts fails to replicate this fear. Zswap really is something that should just always be enabled.
I saw this also repeated at Apple (not Zswap since not Linux, but similar idea of compressing pages) and Android.