Hacker News Reader

Comment by OGinparadise | Hacker News Reader

The main problem is that some people decide what's good and what's not online and paint with the broadest brush possible. Spamhaus, sadly I say, is used by a lot of providers as gospel and a lot of innocent sites are hurt.

View on HN

If you ever tried to handle any mail server at all, you would recognize that you have choice in using spamhaus (or any other DNSBL).

These people have put in place a high quality method to discriminate spammers. I've been around since their beginnings, and their list has been incredibly successful (very high quality) for me, compared to njabl and other "dynamic" lists based on honeypots, or backlash entirely (say hi to spamcop).

You would also recognize that you can just as well tag the message with "likely spammminess" for use along the chain, and people would still complain that your "legitimate" message was tagged as spam by SOMEBODY, while you wouldn't complain if it was tagged as spam by a learning algorithm.

In short, people would complain anyway, except that spamhaus is doing real damage to the spammers (as in "the mail really didn't go through") and reducing their revenue, and thus forcing them to come out which such measures. Not that they will accomplish anything anyway. Spamhaus helped stop a lot of known/professional spammers, and I applaud them for that.

I do not have any choice at all about what spam filters the recipients of my email may be using. I have never had this problem personally, but there are many, many accounts on webhostingtalk.com of IP ranges being banned Spamhaus without any evidence of spam; of IP addresses that remained banned after ownership changes hands and other problems. There are always two sides in every story. On balance I think Spamhaus is doing a very good and necessary thing. I don't know about this particular case, but I've read accounts of what sound like very reasonable grievances.

I had a similar problem and never found out exactly why it happened.

The hypothesis I came to was that we weren't using SPF records on the domain associated with our IP address for a long time.

Some spammers were taking advantage of this by sending emails from different IP ranges with the From: header spoofed to be from our domain.

So Spamhaus blocked our IP address on the grounds that spam filters would also be able to confidently block anything appearing to originate from a domain name that resolved to our IP address.

It's extremely unlikely that spoofed headers or the lack of an SPF record would get you listed on an RBL, especially Spamhaus. I can't guess what happened in your case, but somehow your IP address obtained a bad reputation or was unlucky enough to be in a tainted block. FWIW, the very first thing I do after getting an IP allocation is run an RBL check on it and demand a replacement if it's listed anywhere.

Yes, it was a fairly weak hypothesis. OTOH we got on several RBLs a number of times and managed to get taken off them. Once I added an SPF record it hasn't been a problem since.

Didn't use SPF to begin with because there was a large number of hosts legitimately sending mail for the domain and it was a pain to get all of the IP numbers for various crazy reasons.

I've been designing honeypot/traps as triggers for mail filtering infrastructures for years, and this is very hard to automate process. It started like something that you could watch from time to time in the late '90, maybe slab a DNSBL or two, but right now has become a bloody nightmare. I remember when at some point almost everybody started to "reinvent" greylisting out of convergence, even before it was called as such. Reading nanae (via NNTP) was always a good read.

You constantly have to check if there is a chance that spammers noticed your honeypots so that they can avoid them or use them against you as well (the bigger you get the more sophisticated these attackers get too), you have to use tagged email addresses that can be linked back to the offenders. Methods to probe address ranges multiple times before validating them, and ways to automate the unlisting as well. False positives are basically unavoidable at some point, also because spammers themselves like to rotate their addresses based on their previous owners or known datacenters that are "too big to be blocked" wholesale for this exact reason. If they had a chance to know one of your trigger addresses, a common practice is to generate spam from a "safe" range into the trigger address, in an attempt to generate a false positive and thus, of course, backlash. It's sickening.

Exchanging digests of message contents among multiple server cooperatively became a good indicator of spammyness (vipul's razor), though you would catch bulk emails in the process, and spammers quickly adapted to random email contents so that the method became quickly ineffective.

The real problem here is that these assholes don't care as long as they can deliver the message, that's the only metric they have and care for. Maybe you don't care for it, because you can then use filtering later, but that's a huge volume of trash that needs to be shoveled around. I actually witnessed many cases in organizations bigger than a hundred eployees where several servers were used 24/7 just to churn messages through "dspam" or similar filters before delivering to the final mailbox. This is a huge cost in terms of measurable power wasted for a couple of assholes.

You seem to be taking the line of the attackers' spokesman, who accused, rather hysterically, Spamhaus of deciding what goes on the internet. Of course, all Spamhaus does is supply a list of hosts who are sending email spam, and other things like lists of dynamic IPs. Sounds like this hosting outfit was making money hosting spammers and their business is threatened by legitimate countermeasures.

You seem to be taking the line of the attackers' spokesman, who accused, rather hysterically

Ummm, my ISP IPs hav been blocked several times for absolutely no fault of mine. I have a shared IP for browsing and turns out that cloudfare has blocked them. I also had issues with my sites, the IPs signed to me were blacklisted.

I understand that no one is forcing usage of spamhaus db but it seems unfair and white-listing is near impossible.

He's not the only one to do so. Spamhaus has engaged in some shady behaviour; even pg wrote about it once:

http://paulgraham.com/spamhausblacklist.html (2005)

I wanted to believe him. But before I could reply to his mail, I got first-hand evidence that the SBL has in fact gone bad.

As of this writing, any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam. Why? Because the guys at the SBL want to pressure Yahoo, where paulgraham.com is hosted, to delete the site of a company they believe is spamming.

EDIT

Wait, there's more!

http://paulgraham.com/sblbad.html

http://paulgraham.com/spamhaussbl.html

"want to pressure" - I applaud PG for still staying so civil after this stunt by Spamhaus (and it was not the only one, e.g. they also blocked nic.at in 2007) - I am more direct: Spamhaus blackmails others to get what they want. People who blackmail others should be send into jail, even if their actions are useful. The end doesn't justify the means.

any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam.

Impossible. The SBL lists only IP addresses; there is no content filtering at all.

http://www.spamhaus.org/sbl/

Furthermore, there's a lot of FUD in this thread about Spamhaus listing people who don't emit spam. IF this is true, then Spamhaus would have an unacceptably high false positive rate, and we would be able to observe this. In reality, Spamhaus has the lowest FP rate in the industry. Occam's Razor suggests those who claim to have been wrongly blocked are mistaken about the reason for their listings (if they ever existed in the first place).

You are incorrect. (Well, you're correct that Spamhaus doesn't filter content -- but they don't filter anything, they publish lists that various filtering software uses.)

http://www.spamhaus.org/faq/section/Spamhaus%20SBL#270

I hear the SBL can also block domains, how? What is "URIBL_SBL"?

Yes, the SBL can also be used as a URI Blocklist and is particularly effective in this role. In tests, over 60% of spam was found to contain URIs (links to web sites) whose webserver IPs were listed on the SBL. SpamAssassin, for example, includes a feature called URIBL_SBL for this purpose. The technique involves resolving the URI's domain to and IP address and checking that against the SBL zone.

And of course they also have the DBL (Domain Block List), though I don't know if that existed back when PG ran into problems.

Do you have a link to the false positive rankings? I'm curious as to how that is measured.

Good point; I think both of our statements are true due to ambiguous wording upstream. I also took it literally, "any filter relying on the SBL" -- I use the SBL (via ZEN) but don't use SpamAssassin. And so my mail servers wouldn't block any domain that resolves to an IP address in the SBL, as described in the link you provided.

As for DNSBL false positive rates, I haven't seen statistics in a few years, and by now they wouldn't be worth much. The only ones I saw were from 2005 or 2007. This one (linked to from the below article) from 2011 doesn't even test Spamhaus:

http://www.spamresource.com/2011/05/dnsbl-safety-report-5142...

This is just my personal experience saying (in 2013) that Spamhaus has the lowest FP rate, which isn't scientific. I'm kind of surprised there haven't been more FP comparison reports of major DNSBLs in recent years. If anyone has a link I'd love to see it.

Reply on news.ycombinator.com