Efnet klines all Hetzner netblocks
forum.efnet.org
forum.efnet.org
That sounds entirely reasonable to me. Hetzner has no idea whether or not the box owner is the suspected hacker (even EFnet assumed the server was hacked) in the same way that EFnet has no idea about whether an Hetzner employee was the suspected hacker. So three possible outcomes: (a) the box was hacked, so sending said info to the box owner was not a bad idea, (b) the box wasn't hacked, it was the box owner himself doing bad things, in which case Hetzner just gave the box owner personal information about his victim, or (c) the box wasn't hacked, and it's a Hetzner employee doing bad things, in which case EFnet just gave the employee personal information about the victim.
So, exactly why are they blaming Hetzner?
The operator passed on personal information without knowing if a Hetzner employee might be the bad guy. Hetzner passed on personal information without knowing the server owner might be the bad guy.
If the operator had contacted the server owner directly to let him know his box was hacked, the result would have been the same.
Why did the operator have the expectation that an uplink of the server wouldn't just forward the mail to the server owner?
If EFNet are going to report the issue then there has to be a trust worthy contact at some point along the chain of network ownership, and Hetzner should have been that point. Quite frankly, it's irresponsible for a company of that size to behave in that way.
I've personally tracked botnets infecting thousands of machines back to command and control servers at Hetzner. Guess what happens when you send off the report? They forward it to the bot herder.
If the log included chat between the malicious user and the operator, the malicious user __already knows__ the operator's nick, so there's no additional information. If the log doesn't include that, then what reason is there to include the nickname of the operator?
Rather than expecting Someone Else to sanitize our information (which fields do you feel are critical to redact? real name? phone number? IP? zodiac sign? Made-up-nickname?), it seems like it's a good idea to proactively redact the information ourself before making that abuse report.
This is ridiculous. Why would an IRC op want to keep his/her nickname from becoming public? On IRC, even IP addresses are public. Hetzners reaction seems entirely reasonable to me, especially if the server in question was "mots likely hacked".
That isn't what they said.
> On IRC, even IP addresses are public.
This may still be true on efnet. It has ceased to be true on many IRC networks. And even where it is, VPNs, shell accounts, tor, and similar methods are readily available to make the IP address worthless.
> Hetzners reaction seems entirely reasonable to me
Please tell us where you work so we can all be certain never to report any sort of abuse to your employer non-anonymously.
How not? The forum seems to be down now but the announcement heavily implied that all that was "leaked" was the nick name of an operator, which they claimed could somehow be used to deduce the real name and address.
Protip: Don't put your credit card number in the abuse report. Just put the logs or whatever is required to prove that there was in fact abuse from that server.
On top of that, though, forwarding the complaint to the server owner just makes a mess of things, because now the server owner can target you and make your life hell. They can send you threats, DDoS your mail system, flood your inbox, etc., all the while knowing that Hetzner has their back.
It's like if you saw someone assaulting someone in an alley and called the police, and the police took the information you gave them and gave it to the person who assaulted them. Now they know who you are, and they can come after you too. Now your only real solution is to just not report it when someone does bad things, because you know it's not going to get fixed and it's just likely to cause problems for you down the road.
I don't think that this has to do with Hetzner needing criminal business but rather with Hetzner not wanting to shut down an entire server if a part of it has been breached. Forwarding that complaint in it's entirety is definitely not best practice however making such allegations is neither.
And EFnet looks wronged. But also looks bad for spreading allegations against Hetzner of profiteering from criminals without much more substantial backing.
Sticking to the facts of the situation would have produced a more powerful, and damning condemnation.
I'm not sure that is something to be proud of.
The problem with script kiddies is that they're too lazy to create anything of their own and thus cannot emphasis with how hard it is to set up and maintain a free public service. So they don't have any second thought about destroying other peoples hard work. Plus the pseudo-anonymity of being online and the ability to not look your victims in the eye make such crimes easy for even some of the more ethical kids to shrug off.
The only way to have an open internet where everyone is free to create and contribute is to rat out those who seek to destroy it. People like the aforementioned might enjoy all the perks of an open internet, but their actions have the opposite effect.
So I'm not only in favour of reporting abusers, but have actually reported attacks on my own servers to their respective ISPs.
I mean it's only been a big enough problem for 20 years now that you can't even host an ircd on most standard hosting contracts.
This is a deliberate choice by EFnet, it's part of it's culture and bravado.
You won't see this on Freenode.
This isn't really about that though.
Not too long ago there was an ircd exploit that killed servers and split 100% of efnet, which had never happened before. Not to mention its down 50% of its user base from a few years ago. Tough times for efnet.
All servers on the largest IRC networks are very well connected to the internet and servers that are not, wont be able to last long on these networks.
Some IRC servers wont allow you to create new IRC channels during a network split, so an attack where the network is being split constantly can cause quite a lot of annoyance for the rest of the users.
An IRC nick is not a mask one hides behind. It is an alternative identifier for a real person whose other personal information may be locatable.
as an operator of a large IRC network I am forced to keep my real name off my employer's websites, websites like github and my home address out of the public record (director's addresses and such like)
it's a major nuisance
That leads us back to the pseudonym discussions: Not every name that isn't on ones state-issued id is anonymous. With mine, even without clicking on my profile, you can easily find out my name and yes, my address as well.
If I connect to an IRC network (mostly freenode, or gnome/gimp) I'll be online as darklajid as well. I ran a tiny irc network a couple years ago and used the same handle. For me that's not at all hard to believe.
I can totally understand Hetzner for just forwarding abuse complaints to the client (for root servers, the hoster usually has no "emergency ssh keys"), so the faster the original owner of the server can boot out the hacker, the better. At least it's better than disconnecting the customer from the internet entirely, especially as "haxx0ring" a server is damn easy these days, given the numbers of aged Wordpress installs alone. Also, a server owner who knows his server can support me as a hacking victim better than the hoster support who often knows nothing about configuration details, OS, disk encryption etc. on the server.
But I also understand EFNet, that their emails got blindly forwarded is bad, too...
Removing confidential information from such an email should take ~2min, I think that’s time well-spent, especially given that email per se is not a real-time medium.
Also, it places the burden of dealing with spam on the customer, so that the hoster doesn't have to do spam-filtering.
"[...] someone at Hetzner chose to forward
this complaint to the actual abuser him/herself. [...]
Unfortunately, according to trusted sources
(ex-employees) of Hetzner.de, this is policy and not an
exception."
In 2011 there started to appear pornography when I searched for my name in image search. The reason was that a stackoverflow.com scraper showed my answers (which I post under my real name) but my profile picture replaced with porn. I have no idea why he replaced the profile pictures in the first place, but anyway.
The scraper site was hosted at Hetzner. I phoned Hetzner. They told me to write to abuse@hetzner.de, which I did (in German). I received an auto-reply but nothing else happened. I tried to reach their phone support (09831 610061) several times during their business hours (Mo-Fr 7:30-18:00 Uhr) but no one picked up. I wrote several other mails to Hetzner but never received another reply.
After about seven months the scraper went down but I have no idea if it had something to do with my complaints because I never received any feedback from Hetzner.
I have know Idea who was behind the site. They could have found out about me anyway because I posted this publicly on meta.stackoverflow [1], but still: The thought that they might have learned about my identity through Hetzner is discomforting.
[1] http://meta.stackoverflow.com/questions/81872/copycat-site-c...
Various *-lines for IRC: http://en.wikipedia.org/wiki/IRCd#Configuration
D:Lines are less resource intensive obviously. Not much difference for the client though :)
When contacting ISPs some people attach contact information so that the ISP can request more information, maybe even make a call to verify identity and discuss the matter to help resolve it.
This doesn't mean that the information is intended for the abuser himself. I'd expect the ISP to send anonymized information/questions to the owner of said service, and not just forward "everything" (especially without discussion with the reporting party).
Reporting DDoS and abusers risks putting a huge target on yourself, your company, your servers, etc.
That is what the issue is about.
If the information within the headers is that sensitive, the headers should probably not be forwarded from efnet to hetzner. At its most basic their servers are attacking efnets and so should be considered hostile.
By all means pass on contact information to resolve amicably - but don't leave sensitive information.
There's no reason to pass on irc nicks or sensitive email addresses - what use are they to hetzner? I have the same expectation of efnet as you have for the ISP.
If the issue is that efnet is being identified as the reporter to the rogue attacker, then that is not what is stated in the efnet admin's text.
You seem to misunderstand the situation.
I can understand why you might not want sensitive email forwarded to the abuser, but why would you send that information in your initial complaint? For all you know, the person you're reporting to is the abuser.
I know that if there is any problem, they will forward the problem to me as soon as possible, and they are not going to take the law into their own.
The original complaint was something autodetected by their own system, mailed to themselves. The original complaint was attached, header and all. Something about malware on an IP similar to mine.
Also, when will this be in effect, the server i tested from had no problems connecting.
Posted on behalf of Silence, EFnet admin:
Dear users,
EFnet has always been a network that promotes freedom of speech. One of the core pillars of a free virtual society is trust. Trust not only amongst ourselves internally, but an undying trust in the companies that allow their users to connect to our wonderful network. We have survived over two decades, in a world that is increasingly image- and video-based. IRC can offer neither of those. IRC is based on ideas. Ideas that are exchanged in text. With text, as opposed to images and videos, one has to be put extra effort into the subliminal, the meaning, the message. This has been our catcher in the rye, and we intend to protect this content-based communication form, for as long as it is appreciated by the hundreds of thousands who every day turn to IRC for philosophical debates, dating and just about anything you can think of (I’m sure a lot of the things in that last category does not belong here in this text, but you get the picture!).
We rely solely at the goodwill of others, as is the case with most things worth saving. There is no money to be made. We all do this for free. Sure, some companies might have benefited from a small level of advertisement, attracting customers to their products. But all in all, it has mostly been an uphill battle against enormous attacks, sometimes exceeding 75Gbps of DDoS. This has made it impossible for all but the largest organizations to host a server on our network, or any other large virtual society. We are Don Quijote and the weather mills are often winning.
One of our key strategies is to preserve a close relationship with the major Internet- and Hosting Service Providers, as those are the networks that our users connect through. This has worked very well due to our personal involvement with a lot of said organizations. If we find an abuser on IRC, we try to not only ban him or her, but also to contact the provider so that the problem is handled at the right end, often with the involvement of law enforcement, as was the case with Kevin Mitnick, t0rn and a lot of other well publicized hackers/crackers.
Unfortunately, the past few days many EFnet servers (and more are following) have had to ban an entire ISP, which has not happened in over a decade, if not longer. Naturally, something extreme must happen for this to be even considered. Almost always can we find a solution through the use of good old fashioned communication. Alas, not in this case. Well, here is the story (to the best of my knowledge):
One of our EFnet operators located a server that a client was using, the server was most likely hacked and/or used for illegal activities. As IRC is often a playground for these people to use, before moving on to more serious targets (where they can make money through extortion), we take this extremely seriously. Because of the serious nature of this, our operator sent an email to Hetzner.de, a German hosting provider, to help them lower the abuse of their servers, as well as ours. This is usually a fruitful symbiotic relationships, where both parties stand to gain.
However, the big difference between this case and all the other thousands of cases we have handled in the past, is that someone at Hetzner chose to forward this complaint to the actual abuser him/herself. This might seem fair enough, as anyone accused should be granted the right to defend him- or herself. However, the email also contained sensitive information about who this operator was, including nicknames (from which names can be derived, and thus, also, home addresses). We know what an impact this can have on your social, not to mention your professional life. We have seen people lose their jobs, after constant attacks and we have also seen companies lose money that is hard to fathom, considering this is still just a simple chat for friends. This is a fundamental breach of that mutual trust that has allowed us to accept clients from Hetzner to use our network - free of charge, just like we do with anyone else wanting to connect.
This a give and take network, where mutual trust is vital for our survival. We are maintained by the community, and we exist solely for the community. Hetzner.de has broken one of the most fundamental aspects of any report of criminal activity or suspicion thereof; source protection.
I expect us to get attacked now, which will result in a lot of work for the company kind enough to donate money and time to continue to provide us with servers, in an era where almost everything else would be more profitable. But this is an ideological problem, more than a financial one. We have been attacked before, and we will again. We are prepared. But these preparations rely on the fact that we know who the enemy is. Hetzner.de has made that impossible.
As a result of this, we have decided to ban all Hetzner IP ranges (both IPv4 and IPv6) from our servers. It seems other networks are following, and I know QuakeNet has published a similar statement. We simply do not want anything to do with a company that values money over source protection and integrity. Some may argue that this was a one time mistake, and that we should not jump to conclusions so fast. Could this have been a mistake? Sure. Does it matter, given the consequences this could have had for this operator’s personal life and health? No. We do not appreciate cowards that would rather see someone else hurt, than take their responsibility.
Unfortunately, according to trusted sources (ex-employees) of Hetzner.de, this is policy and not an exception. They have realized they can save money (by limiting attacks) by redirecting the attacks back at the person reporting them. That way, the hacker/cracker/kiddie using their services will not cancel their contract with Hetzner, and in return Hetzner will remain protected. Left are those of us that work for free, and who will continue to do so, for as long as there are honest, reliable companies out there, willing to go the extra mile to protect the freedom of the Internet, and, above all, freedom of press and source protection.
Questions on this matter must be directed to Hetzner.de, as our involvement in this situation is over. This has been their decision based on questionable methods. It is unfortunate for them that they got caught, but it is good for the sake of the free Internet.
Sincerely yours,
Johan Boger, on behalf of EFnet and anyone else believing in integrity, source protection and a free Internet.
> ...
> the email also contained sensitive information about who this operator was
> ...
> This is a fundamental breach of that mutual trust
> ...
>Hetzner.de has broken one of the most fundamental aspects of any report of criminal activity or suspicion thereof; source protection.
Excuse me? How is this in any way Hetzner's problem? The only "fundamental breach" of trust I can see here is the one that occurred when efnet gave "sensitive information" on its own operators to a third party at all. It sounds to me like efnet is the one in need of a lesson on "source protection." This reads more like a "we screwed up bad and are about to divert blame as hard as we can" letter than anything else. But I'm more than ready to listen to any further context that can be given to this.
Instead of trying to fix this like normal persons, you want others to work for you by free.
Who gives you the role of a judge in this case, to even consider that the actions from that ip where illegal.
If they were illegal why don't sue? If you ask a lawer the first thing he will tell you will be to talk with the other parties before starting legal actions.
Your local ops team notices a lot of SSH break-in attempts coming from Sprint's network (for example). They forward the logs on to the abuse team, who reach out to the abuse team at Sprint. In the same way that a cross jurisdiction investigation would be handled by two police departments working together, the two abuse teams will work together to investigate and remediate the problem.
Now imagine if the Seattle Police Department contacted the Chicago PD about an ongoing investigation of a jewelry heist, and the Chicago PD says "we are too busy to deal with this" and forwarded it on to the thieves so they could respond directly and help the Seattle PD with their investigation... Now the investigating officer is compromised and at risk. Sure, he was a known as a cop to the bad guys before, but now his enforcement actions have made him a direct target.
This is a system that has worked for 20+ years. We have our own mailing lists, meetings and conferences, heck even a private direct dial phone system.
They do not mention that these emails get redirected to customer!