Perhaps the author of DuckDuckGo[1] did not know about those limitations...

As far as I know, this search engine uses both own crawlers and some external sources of data. One person team did it.

----

[1] http://duckduckgo.com/