Ask YC: applications of bots/spiders/crawlers

Vulnerability scanning would be one potential example. An intelligent bot can better probe and respond to services running based on OS/Version returned.