Hacker News Reader

Through Bing Webmaster Tools, I can see pages being crawled, but not indexed. Bing has the sitemap and all the URLs. The robots.txt allows all crawlers. Even if there was an issue there, it would be on their end, and they should be the one fixing it. The result is the same: a mysterious disappearance from their SERPs. And no way or clue on how to fix it.

View on HN

No doubt they'll attempt to crawl and their webmaster console will say so, I just think it's important to verify the request actually made it to your server via your own server logs in cases where you're using a WAF like Cloudflare, just to ensure that Bing saw what you intended to serve them. If they're reporting 4xx errors, it could just as easily be a WAF giving them the code, or your server but judging by your response, it wouldn't be your server. The server logs would explain away some of the issue.

Indeed. Thanks for the help!

Reply on news.ycombinator.com