Hacker News Reader

Comment by chimeracoder | Hacker News Reader

> Is there a reason the same techniques used in E-Mail aren't applicable to Twitter?

Twitter relies on very low latency - ie, once you tweet something, if it a whole minute to appear in your friends' timelines, it could already have lost much of its value.

Lots of spam reduction techniques introduce latency to levels that are unacceptable to Twitter's use case.

I'm not sure why it didn't catch these, but I can imagine why the same techniques aren't applicable in general.

View on HN

I doubt if that's even true, because Twitter's tail latency on API calls is in the hundreds of seconds. They do not operate a low-latency service (and they don't operate a high volume service either, by my standards).

I think the real problem is not latency, but simply they don't have the signals needed to differentiate spam from not.

Where did you get the latency data from?

Speaking as a user, when someone sends me a reply, I expect to see it in seconds via the streaming API so I can have a short IM-style conversation, and I consider this a nice benefit to using Twitter (rather than, say, email), if not totally essential. Hundreds of seconds sounds bad, but if that were the norm, it would be a serious problem.

Sure, but couldn't they run something that cleans up already posted tweets? That wouldn't introduce any latency while posting but would still (eventually) get rid of them automatically and hopefully pretty fast.

I think they are. Most of the search results I'm seeing are only up until a minute ago.

They do – they deleted a tweet from my timeline.

If my twitter account was compromised and used to send spam, I think the way I'd discover this is by seeing the record of tweets sent by the false "me". If those are being culled, then how will I know that my account has been compromised?

You can do pretty aggressive spam detection in way less than 100ms.

Source: I wrote a system that can do it in less than 10ms at Grooveshark.

any way i could pick your brain for 20 mins? we're dealing with a lot of it at whisper... would buy beers or ... whatever you want.

hey I work on anti-spam at FB. Happy to talk sometime. facebook.com/mkjones or @fisherrider on twitter (lol).

You can learn a little about how we think about securing login here: http://vimeo.com/80460475.

thx, will reach out!

Shoot me an email james@grooveshark.com. Love to chat!

I've never timed it, but I do try to do realtime-spam detection on blog/forum comments:

http://blogspam.net/

Although it operates at a much smaller scale than most of the things linked to here it does work pretty well.

Reply on news.ycombinator.com