Comment by ChuckMcM | Hacker News Reader

Comment by ChuckMcM | Hacker News Reader

Parent Full thread

That is a great story, at Blekko my first attempt at getting rid of people trying to exploit the index was to ban any address that did more than 100 queries per second, that got a bunch. Then they start showing up again, at 99 queries per second :-). So I lower it to 50, same pattern but now 49 qps, then set it to 10 qps, they would limbo in under the wire again. They would also start using bot nets to send their query from different PCs in different places (all the same query though, typically looking for ecommerce or wordpress templates with known exploits) We would nail them by finding all the machines that made the same query, then we could just start killing any query that asked for more than page 10 of the results, etc.

For a while I pondered if I could use my ability to change their behavior by changing my rules, to get them to actually create a set of behaviors that would unmask their C&C servers but not enough free time for that.

You could start sending fake results after they exceed the ~100 queries, they will probably stop exploiting it.

This is a good idea, actually -- make it impossible to distinguish a banned request from a succesful request. At least that could delay the discovery period, as would randomizing the request limit.

I've wondered if a web server or proxy in something like Erlang would be ideal there, where there is little server-side penalty in dragging requests out progressively longer.

With the lightweight processes, the cost per should be very, very low, and basically make it easier to Turing tarpit the scammers.

Yeah, "hellbanning" scammers is a good idea, instead of giving them a simple 401 code. Increasing the cost of their scam is the best mechanism.

There are public records for businesses. I just learned from another comment that locksmiths are state licensed.

Wouldn't those data be useful for fraud detection and prevention?

Are the records all of the following: 1) structured 2) available via web API 3) consistent across jurisdictions 4) reasonably up to date

I'm guessing they're not. Google has had hundreds of really smart engineers with an effectively unlimited infrastructure budget working on Maps spam for a decade. If it were easy to solve once and for all, they would have.

There are at least a dozen companies that aggregate and resell that data. I'm familiar with LexisNexus (nee Seisent). I'm certain Google could do it too.

Reply on news.ycombinator.com