Hacker News Reader

Isn't this just the "link preview" feature, that is enabled by default in outlook?

Many email clients generate link previews so that they can display a thumbnail of the webpage. Would seem to be necessary to filter out those referrers from the validate link

View on HN

Essentially yes - but "previewing" link and sending that link to Bingbot to crawl and index is another matter.

Imagine you send someone a "private" link to a file...Bing sees that and indexes it for the world to see. Not cool.

“Private links” should be covered by robots.txt. The only case I see this happening is for those “anyone with link” shares and those are easy to cover.

AFAIK, most of those "private" links are actually just "unlisted" but they're still public. I'm sure Bing is indexing those too...

Anything private should ideally be put behind authentication. If that isn't possible, than robots.txt. Search engines are _meant_ to index everything that is publicly accessible and not blacklisted on robots.txt.

Is this actually the Bingbot crawling for the index, or do they just use the same bingbot code to generate link previews?

If I have a well-tested crawler sitting here, I don't see why I'd write a brand new one just to fetch previews in outlook...

Reply on news.ycombinator.com