Hacker News Reader

Comment by jgalt212 | Hacker News Reader

Yet, advertisers pay for 100% (or very nearly that amount) of ads displayed--regardless of the client.

Dunno. I assume quite a lot of bots don't execute Javascript and therefore don't get to "see" a lot of ads anyway.

that's good point. It would be nice to see someone who does not have a vested interest in under or over reporting the number of bots on the web take a good crack at this number.

Per spider.io, bots are ripping off advertisers left and right. That being said, scared advertisers are the target market for their products.

http://www.spider.io/blog/

I've got no investment either way. My nginx log has 19,333,928 entries ... when I do grep -Ei '(bot|spider|crawl|aggregator)' on it I get 2,609,874.

That means that bots who declare themselves make about 13.5% of requests to this site. The robots.txt has no restrictions.

I don't know if that's helpful.

That doesn't really tell us anything about whether or not they trigger javascript. Bots triggering javascript will inflate advertising and js-based analytics numbers, which would be more problematic.

Bots that don't trigger js, however, are more a drain on bandwidth than anything else.

I seem to remember from a job I used to have as an analyst that advertisers have a few ways to discern human and bot traffic. They don't pay for traffic that is known to be from bots. However there are clearly some bots that aren't accounted for this way, and some that are designed specifically to trigger ads.

I'm pretty sure that is built into the price.

That tidy solution only works if the fraud rate on the ads you place is less than or equal to the industry presumed/priced-in average. It seems that there are more than a few ad networks around these days that specialize in non-human traffic. As such, advertiser realized fraud rates can vary greatly from this average.

My company right now won't purchase RTB ads that display outside of walled gardens so that we can 1. manually verify that the ads are showing. 2. Minmize the number of bots seeing the ads (presumably it's hard for bots to infiltrate Facebook or the WSJ than Buzzfeed).

Reply on news.ycombinator.com