Reddit is currently experiencing a malicious DDoS attack | Hacker News Reader

Reddit is currently experiencing a malicious DDoS attack

twitter.com

Reddit also got hammered pretty hard by legitimate traffic last night. There were many thousands of people in the Boston live update threads hitting refresh over and over all night. The admins actually locked the first update page because they said that so much traffic on an article with so many comments would crash the site.

If there's a malicious DDOS attack then this surge in legitimate traffic likely compounded the problem.

The update threads: [1] http://www.reddit.com/r/news/comments/1co395/live_updates_of... [2] http://www.reddit.com/r/news/comments/1cnwms/mods_removed_th...

Where the updates moved when reddit went down: [3] https://twitter.com/JpDeathBlade

The traffic surge from the Boston incident is extremely marginal compared to the DDoS which we're facing right now. It is far and above any form of possibly being compounded, unfortunately.

But I wish it was :) Compounding I can deal with, astronomical is harder.

I hope this is a retaliation for the abhorrent behaviour of reddit users in blind smearing of innocent individuals in an attempt to prove the worth of social media/the internet/their "community". Information is dangerous when it is placed in the hands of these clowns and reddit needs to address their problems quickly before they ruin more lives.

http://www.newstatesman.com/world-affairs/2013/04/reddit-bos...

Actually posts containing speculations about Sunil (missing student) were instantaneously deleted (as groundless and potentially harmful) , at least before name appeared on police scanner, at least on r/findbostonbombers. Info was spread somewhere else, because new people constantly were creating ones.

before name appeared on police scanner

Is there a recording of this? Because it sounds a lot like it never happened.

It didn't the student's name was mentioned by a friend of his on Twitter who after looking at the photos released by police said that they thought there was a resemblance between the suspect in the white hat and her friend who had been missing for a month after dropping out of school and becoming depressed. There were several articles about the individual already on the web predating the bombing because the FBI had been contacted by the guy's family to help find him. The name mentioned on Police scanners was a different person also incorrect.

No, this happened. A purported classmate of Sunil's had tweeted about the resemblance to Suspect #2 about 14 hours ago:

> @christiiinaaaxo @jaredleesmith @thejman218 Looks just like a kid from my area that went missing exactly a month and has yet to be found.

> I mean maybe it's a total coincidence but I'm more than a little freaked out by the resemblance. #Boston pic.twitter.com/Qsd6rj5v5l

At some point, the Internet ran with her speculation:

> It makes me sad that people are going to the Tripathi family Facebook page and writing nasty things. That was never my intention

And at around 4-5 am, the Boston police scanners said they were looking for Sunil. I remember this because I fell asleep at 5am thinking that this bizarre story involved Sunil and then woke up at 8 learning that two completely different suspects had been identified. From the twitterer:

> So if what I'm hearing is correct, at 3 AM when the police scanners said they had their suspect and it was Sunil, I was wrong.

The tweeter's account is now protected but she hasn't deleted her tweets. I think everyone, including her, has learned a little about making well-intentioned speculation on the Internet. But apparently, even the Boston police had been fooled (at least someone in the BPD was, which is why police scanners should not be taken as gospel)

the Boston police scanners said they were looking for Sunil

Did you hear this yourself, or did you read it? That's really what I'm asking. I know many people were listening, but it's hard to tell what people heard first-hand and what people simply re-tweeted.

For my own peace of mind I'm trying to distinguish between two possibilities: a) that the names were not heard on the scanner at all and whole story was either a mirage or an intentional hoax, b) that someone's erroneous tip to the police was mistaken for its own confirmation.

This is an excellent point. No I did not hear it myself and would not have been able to tell what was a first-hand account of hearing it from merely retweets/reposts of that information. And whoever originally heard it may have heard chatter that said the names, but not in the context of being actual suspects.

Actually Sunil wasn't the main person who had been accused on Reddit

Can you show me some examples of abhorrent behavior? In the few reddit threads I skimmed about the bombing everyone was surprisingly mature, especially when pointing out 'suspicious' people.

isn’t ‘Pointing out ‘suspiciuos’ people’ on a public forum rather immature by itself?

For the most part, they were pointing out suspicious pictures, in the wake of a nearby large-scale criminal event. Not suspicious people by name.

What were the pictures of?

Would the pictures, or the people in the pictures, be suspected of being bombers? Were the police looking for pictures or for people in pictures?

Not in my opinion. They were just people trying to catch the bombers. No one was jumping to conclusions from what I saw (maybe some did but they were downvoted further down than I scrolled), rather they were just pointing out scenes that looked suspicious to them so that the police could investigate further. While there have definitely been cases in the past where a line has been crossed, I don't believe that to have happened this time.

Have any lives been ruined by Reddit? Do you have proof of this?

> I hope this is a retaliation for the abhorrent behaviour of reddit users

You don't like vigilantes, but you hope vigilantes are punishing reddit users?

Quis custodiet ipsos custodes?

"Who watches the watchmen" would have been simpler and spared me a google search.

But I wouldn't have learned a cool new latin proverb.

Although it should be noted that the Latin itself is a translation, since this quote is attributed to Socrates and would have originally been in Greek.

According to Wikipedia it's (incorrectly) attributed to the Socrates character in Plato's Republic. Although a related idea is discussed in that book, the saying itself is attributed to Roman poet Juvenal, which was discussing marital affairs.

Also, I wouldn't have gotten that feeling of smug superiority for having read Terry Pratchett.

But then how would we know that facorreia is smarter than we are?

awesome fodder for a reddit comment chain, let's continue it there

Quidquid latine dictum sit, altum sonatur.

Being denied cat pictures and trite memes is a reward, not a punishment.

of some reddit users. FTFY. Let's not smear everybody in your blanket statement.

The voting majority of reddit users.

while I was quite shocked by the witch-hunt and amount of 'suspicious until proven innocent' I saw in that subreddit, in fairness to reddit as a whole that is one small subreddit among thousands.

Also, some of the highest upvoted threads and comments appear to be warning others of the potential negative consequences of their publicly pointing out everyone with a backpack or different coloured skin.

Some small subreddit goes off the rails every few months.

Reddit as a whole needs a better answer for that than just pointing out that it is just some small subreddit.

(I don't pretend to know what that answer is, and I guess it does have more to do with people choosing how to participate in online discussions than it does reddit, but reddit is well positioned to push that participation in a better direction)

Last time I checked, the find Boston bombers subreddit had ~5K subscribers. r/humor currently has ~3.6 million. Please check your facts before making any more claims...

As opposed to a non-malicious DDOS attack?

Sorry about the snarky remark, but the headline does feel a tad redundant.

Non-malicious DDOS attacks might be considered those where the traffic is genuine, like a normal website reaching the frontpage of reddit or hn.

By definition, DDOS, are malicious no? Otherwise, the site is just having high traffic that it cannot handle. If my blog goes down after being on the front page of HN, is that a DDOS?

I think it's more the term "attack" that indicates maliciousness.

Getting Slashdotted or Hacker Newsed is definitely a distributed denial of service, but it's not so much an attack

I don't think it's appropriate to use the DDoS acronym when not referring to an "attack". It's non-standard and will only create confusion. Every "source" online describing a DDoS uses it explicitly to refer to an attack.

By definition: "[DDos] is an attempt to make a machine or network resource unavailable to its intended users.". When I see "DDoS" I always thought it was malicious...

Was Service Denied? Was the cause Distributed?

People sometimes make mistakes. If those people create and distribute consumer level routers those mistakes can be severe.

(http://pages.cs.wisc.edu/~plonka/netgear-sntp/)

Non malicious DDOS happen all the time, but mostly reddit does it to other small sites that people discover and reddit ends up overwhelming the servers. Still technically a DDOS, but non malicious. Also during Obama AMA, site was in shambles because of itself.

I heard someone describe a non-malicious DDOS attack (in the case of high traffic) as a website being cuddled to death.

"Typical HN Top-poster - up in arms about one point they care about, misses entire point of article."

https://news.ycombinator.com/item?id=4780475

Don't confuse missing the point of the article with not caring about the rest of the article.

I had a chuckle too about "malicious". A DDoS attack is a DDoS attack. It's like hearing about a "malicious stabbing."

But whatever, there are DDoS attacks every day on someone.

Yes, as opposed to a non-malicious one.

The best example off the top of my head was a D-Link firmware update which added an NTP server operated by a third party for "public use." This wound up increasing the NTP server's traffic by 90% and costing them over $8K/year in additional bandwidth fees. See [1].

Then you have so called "slashdotting." Or linking to a small web-site with an interesting story and overloading it with legitimate traffic until it goes down. See [2].

[1] http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse#D-L...

[2] http://en.wikipedia.org/wiki/Slashdot_effect

Getting a slashdotting is not a "DDOS attack" under any definition although the practical consequence is the same. If the resulting traffic overload is caused by a (massive) router-configuration error or similar mishaps, it's also by definition not an attack.

Sorry about being picky around semantics, but still: No point in throwing words around to the point where they become completely pointless.

Well ... It's definitely a DDOS but you're right about the attack part.

There is no inherent requirement that attacks need to be intentional. It's possible for a software glitch to launch a missile attack. Attack: An aggressive and violent action against a person or place

I took the "malicious" as being "severe" so didn't really find the wording redundant.

Could this have something to do with reddit's co founder speaking against CISPA and calling out major tech companies? [1] Considering reddit played a big role in killing SOPA, CISPA backers feel threatened?

[1] - https://news.ycombinator.com/item?id=5570670

I think it's far more likely that someone decided to mess with the party all the Boston observers are having on Reddit.

It's all idle speculation, though.

Yes, reddit was up for a bit and I saw two (edit: now four) huge threads with 10k+ comments and live updates from the shoot out in Boston. So maybe they are indeed messing with the Boston live event updates thing on reddit. DDOSing reddit is no small feat, unless you have a botnet at your disposal. Should be more than just some "fun".

I don't have data to back me up other than an anecdotal 'I seem to notice', but doesn't reddit often become unusable during exceptional usage spikes? the Obama AMA comes to mind.

if you look at their numbers[1] for 2012, it's plausible that they are simply overwhelmed during these bursts and they've judged it worthwhile to let these aberrations play out.

for many operations.. moving your availability from, say, 99.9% to 99.99% is simply not worth the cost.

uptime is hard; GitHub has only maintained 99.96% for the past month[2][3] (and, IMO, they're actively trying to improve it)

[1]: http://blog.reddit.com/2012/12/top-posts-of-year-and-best-of...

[2]: https://status.github.com/graphs/past_month

[3]: http://cl.ly/image/0O2S0L3O0l3J

edit: clean up links

edit2: learning \n

But reddit confirmed it's malicious, so the idea that reddit DDOSed itself is out of the question.

I would think it is a malicious attack in response to Reddit incorrectly identifying and popularizing the idea that Sunil Tripathi was one of the Boston Marathon bombers.

Unlikely, why would someone retaliate? Who on Tripathi's side has such power? Right now we are not even sure if the attack is related to boston events at all.

The LOIC does.

We're mostly recovered, at this time.

How certain are you guys that it wasn't the sheer levels of traffic due to the ongoing Boston situation?

Certain beyond any shadow of doubt.

What are the indicators that this is a malicious DDoS attack rather than just unprecedented traffic from interest in the Boston manhunt?

Also, what reason is there for a DDoS attack on Reddit right now?

https://news.ycombinator.com/item?id=5570670

They're slamming over 20x our normal bandwidth with URL containing things such as "boston", "your_gay", etc.

Maybe government doesn't want redditers countering their official narrative regarding the suspects

Seems like you're still down...

now that is probably just getting back to their regular intermittent availability.

You might be overzealous with your bot blocking. I'm being told I'm actually a robot:

    you appear to be a bad robot
    check out the rules for robots. thanks.

Same here, are you using RES or anything else which might increase request volume?

I use RES, not having any issues.

I believe I've had this happen when using a VPN service (though I could be mistaken)

Is there a defense against DDoS attacks? CloudFlare? Incapsula? Does anyone have experience with how well these solutions work?

yah, put a swarm of raspberry pi attack reflectors in front.

In ultra-simple terms you need to have a network capacity greater than the attacker and to identify the attack requests and discard them whilst still honouring valid requests.

That is basically what CloudFlare is.

Add in things like caches to prevent even valid requests from getting to the backend (so you've now added a CDN), and many peers to your network so that an attacker cannot saturate one or two peers... and you've got the essence of CloudFlare (sans features like optimising content for speedy delivery).

Ultimately the best defence to a DDoS is to be able to soak up the attack before it hits the backend, and to have enough spare capacity to keep serving regular traffic.

You can cache and distribute everything save for the valid requests to a dynamic resource (but even those you can optimise). So the whole game from a defensive point of view is to let nothing but valid dynamic requests through to the backend.

The attacker's side of a DDoS is about acquiring network capacity greater than your network capacity, identifying your weaker points (in an attempt to cause a domino effect, if they can take out a weaker peer then a stronger peer will need to do more work and itself becomes weaker). And then sending what appear to be valid requests without triggering an attack on themselves (having a 150 byte request respond with 100KB would wipe out the attackers). Bonus points for constructing requests that can get through to the dynamic resources on the site being attacked (as those are the weakest link).

how is adding things like caches adding a CDN?

I probably should have clarified that by caches I mean reverse proxy caches that can take up the work of serving static resources from the network edge.

The combination of adding caches and distributing those caches is to add a CDN.

You add caches to stop the request reaching a backend and doing the work twice, for optimisation. But in effect they become defensive shields as serving a static file or an in-memory file is less work and can be handled in far greater numbers than doing the work on the backend, and if one cache is attacked users accessing other caches elsewhere in the world continue to get their requests served.

If you then place a cache at every point at which your site is surfaced, for example you use DNS anycast to have your front-end appear to be surfaced from every Amazon datacenter and the closest one is nearly always selected... then you've helped stop requests at the first opportunity and to return them from a place which can handle far greater requests.

You've increase your network capacity, increased the ability to serve valid requests, and you've prevented all of that traffic reaching the backends.

And in doing all of this... placing caches for static resources throughout the world and using DNS anycast to return the cached item from the closest peer... well, you have created a CDN. A primitive one for sure, but it still is one.

This is a very good explanation but there is another side to this issue. (I`m talking about HTTP DDOS)

Basically, DDoS Attacks can be (roughly) divided in 2 categories:

1. Attacks on your server (which usually target your server IP or some other part of your network infrastructure)

2. Attacks on your site (which use bots to flood your site with fake HTTP requests)

As explained above, Network attack can only be countered with strong and flexible infrastructure. The most common solution is a combination of several high-powered servers and load balancing capabilities.

HTTP DDoS attacks are trickier because they're best mitigated by visitor profiling, a technology that can help identify bots from humans and block them while still allowing providing full access to all legitimate visitors. Developing and maintaining such technology is arguably more complicated, simply because it's a software you need to create, not a hardware which you can buy.

Standard profiling solutions include CAPTCHAs and Delay Pages but these will also repel legitimate visitors. (because no one likes CAPTCHAs or waiting for 5-10 extra seconds for page load). Advanced profiling solutions use a combination of behavior and signature recognition, coupled with seamless challenges (i.e. checking for JS support).

CF and Incapsula (where I work) both handle Network DDoS in a similar manner but we have a somewhat different approach to HTTP DDoS.

And yes, while under DDoS (or even without it), dynamic resources can be the "weakest link". This is why WAFs are so important.

I wonder if it would be possible to generate a DDoS attack by reporting that a site is experiencing a DDoS, causing people to flock to the site to see if it is down.

I was on Reddit until 2am (EST) with no issues. Back on at 6:45am with no problem.

Either they've been doing a wonderful job with mitigation, or I'm missing something (I suppose I did miss a few hours while sleeping...). If anything I figured they're getting hammered with EST sunrise/Boston incident traffic...but a simultaneous malicious DDoS attack "beyond any shadow of doubt"? Wow

Total speculation, but: perhaps the police / FBI would DDoS Reddit to prevent the suspect from gaining intel on their operations? The Reddit live stream was all over Twitter...

The chinese are trying to censor reddit because of that article about china censoring the term "censorship".

Fark is down too, 4:21 am Pacific time.

what is impact of this "DDOS" attack on reddit?

hopefully that reddit is over and all reddit-users can leave the internet now ;)

Batten down the hatches, pg, here comes the redditors.

Reply on news.ycombinator.com