Ask HN: Facebookexternalhit behaves like a Spider – Block or not to Block?

Block or not to Block?

facebookexternalhit behaves like a Spider indexing all sites, not actually what was submitted to Facebook for Card data

What originally was supposed to be previews of URLs is now being used to crawl all the internet (based on our logs from different sites). It also sometimes makes up non-existant pattern URLs to crawl more that it can find in it's index.

I assume all the data is used for LLAMA which it hides behind the default facebookexternalhit.

Go ahead, check your site logs.