Hacker News Reader

Comment by FSecurePal | Hacker News Reader

12 pointsFSecurePal19 comments

Blogspam; original and much more informative article at http://jl.ly/Email/v6bl.html?seemore=y .

Blacklists weren't exactly a great solution to start with.

True, but they are the best defence we currently have available.

I would like it if the largest players in email (Microsoft, Yahoo, Google, AOL) got together to organise the creation of an independent not-for-profit organisation which would run a public whitelist. They could set a cut off date after which they would no longer accept email from non-whitelisted addresses.

Email admins who don't usually pay much attention would suddenly notice their mail being rejected, would be forced to register on the whitelist, and would ideally implement the db themselves on their systems.

The difficult part would be deciding how IPs get onto the whitelist and under what circumstances they are removed. And its day to day management of course.

No, they are not the best defense available. They are a questionably effective interim band-aid at best and a useless hack at worst (since they reduce the effectively useful IPv4 space). Other methods such as baysian filtering and sender authentication are far more useful.

I have been an email administrator for an ISP and a University, and I have been active in many spam related forums/mailing lists for many years now, and I completely disagree.

Blacklists catch far more spam than bayes, are far easier to administer and far less resource intensive. Personally, I try to deploy both, with blacklists first in line, then bayes to catch some stuff that the blacklists miss.

"Sender authentication" is a vague term. If you mean verification of the sender address via various DNS lookups, then yes, that's quite useful, but still nowhere near as useful as blacklists.

This is how I see people setting up their spam filters in the industry. It might not be idealistic, but it is effective and wide spread.

There are better methods than just doing a dns check. Actually signing the messages works too, as does whitelisting of ips by the domain owner.

You're describing SPF and DKIM. Both of which can be used by people sending legitimate mail, and people sending spam...

SPF and DKIM are good for preventing some false positives. They're not good for stopping spam.

I would bet that currently, well over 90% of spam that is rejected, is rejected because of blacklist lookups.

Well, for the first two or three years of the problem, the solution will be to blacklist IPv6 in toto. If you're not "good enough" to have an IPv4 address to send mail from, then you lose.

I'm not saying this is perfect or endorsing it, I'm just saying that this is what is going to happen. It's not even an interesting problem until you have actual mail coming from IPv6-only addresses on a routine basis, and that happens well after the IPv6 switchover, which still hasn't happened.

So how will that work for gmail? They can't throw away their entire userbase because they don't have ipv4.

Expanding on what pkulak said, it's the mail servers that send mail, not users. There will actually be quite a long time during which everybody can still reach an IPv4 server. They may even reach it over an IPv6 address, but when the mail leaves "your corporate network" or "Comcast", it can be over IPv4. So even a brand new ISP in a brand new country with brand new network addresses and no other IPv4 connectivity can still have one IPv4 mail server somewhere. (IPv4 addresses aren't so rare than ISPs can't have them, what we can't do is hand them out to every individual on the planet.)

And since in the early phases of the transition anybody wanting to send mail will need to do this anyhow, there will be a long period of time (couple of years) where pretty much all mail coming from an IPv6 address will be a spammer trying to get around IPv4 IP reputation lists. And the solution will be to blacklist IPv6 as a whole, and that solution will be deployed until it becomes too painful. Following that will be whitelisting specific known good IPv6 addresses, which will probably hold for at least another year. Only then will be actually be thrust into the chaos of a network where you can't use blacklisting, and, well, I actually still have some ideas on that front so I'm not sure it's hopeless even then. (But I can't test the ideas against reality until about year 2 of this process at the earliest.)

Again, let me highlight I'm not endorsing this; my first post has been bouncing up and down between 0 and 1 and I suspect there's some people missing this point. I'm just saying, it's what is going to happen. Mail filtering is very much an engineering problem, not a science problem, and you can sing the virtues of IPv6 to the mail server admins all you want but they do not care.

There is mail being exchanged over v6 today. Most of it is legitimate at this point although there are v6 spammers operating already, of course. The volume is still pretty light but it's definitely not zero.

It' still possible to use blacklisting at the ASN or netblock level in v6. I have no doubt that people will do this despite the extensive collateral damage it causes. This type of wide blacklists are a reality in the v4 world also and they cause damage there too.

I expect SPF, DKIM, and content inspection will become more primary as DNSBLs lose effectiveness in a v6 world. Right now you can skip this type of processing on 50-90% of inbound attempts thanks to the DNSBLs effectiveness. It can be a huge cost burden to to process the amount of crap being sent by spammers so SPF is the best option but it can't do the job without the other pieces.

Google needs the ip4 address in this scenario, not the user.

Interesting point. Blacklisting based on the /64 won't work, due to the potential to deny service to legitimate users, so what will take its place?

Also, it's already started: http://mailchannels.com/blog/?p=114 (First IPv6 Spam Message Caught in the Wild)

I don't understand this statement: "Blacklisting based on the /64 won't work, due to the potential to deny service to legitimate users,"

At the moment, we have multiple users sharing a single IPv4 address, so when that gets listed, there is collateral damage.

If we change to having lots of users sharing the same IPv6 /64 and we start listing /64 there is no change to the collateral damange...

If every user starts getting their own /64 to send mail from, then this will reduce the collateral damage, but make it harder to list. In which case, larger blocks will get listed, again bringing the collateral damage back up to the same level we already have it at...

In general, a /64 is being assigned in a similar way to a single IPv4 address is used today. Blacklisting a /64 would be similar to blacklisting a single IPv4 with multiple machines behind a NAT.

If anything this gives us more control. You can start by blacklisting individual v6 addresses, and if you notice an above average number are from the same /64 knock off the whole thing.

Whitelisting. Only allow mail from certified domains or servers. Have a sort of certification/cost barrier system in place to prevent spammers from gaining access. That, or a distributed web-of-trust scenario with authoritative entities like google in the chain.

So you block the /64. There aren't any subnets smaller than a /64 in IPv6, so you're guaranteed to be blocking a leaf node.

This is still miles better than having all of the customers behind large scale NAT, where you risk blocking innocent folks who happen to be behind the same NAT.

What if you were required to have a valid CA certified key to relay messages? Then we could reject connection attempts that came from unauthorized hosts and we could revoke keys for spam relays.

Some hosts send both spam and ham. Some hosts send mostly ham, but are occasionally abused to send spam. One countries definition of spam is different to that of another countries. One persons definition of spam is different to the next. Who runs the CA?

This isn't a difficult problem to solve technically. It is a difficult problem to solve socially.

Reply on news.ycombinator.com