The solution is allow only communications that are demonstrably not spam and not unwanted.
There have been lots of proposals to this effect but maybe you have to put up two cents in a crypto wallet, which the other person can claim if the mail is unwanted. If someone takes your two cents, you remove them from your lists ASAP. The receiver doesn't benefit meaningfully from the 2c so they are unlikely to claim it. The sender is only damaged if a lot of different people claim the 2 cents. Maybe a few domains are default whitelisted even if they don't put up 2 cents, to protect against entities that are important but would be frequently spite-claimed (IRS, courts, and other similar). If you use a provider that doesn't whitelist, fine, but you're going to miss those emails.
Maybe the top 'spite claim' users are published and it becomes difficult to get people to agree to put up money to talk to you.
This reduces spam to the level of junk mail, which is still not totally trivial but is orders of magnitude more manageable.