Hacker News Reader

The odd thing, is that getting a list of all profiles correctly doesn't wor. If you go to Google's robots.txt and scroll to the bottom to find the profiles sitemap is located at:

http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xml

Which 404s.

Luckily, it's quite easy to reverse engineer. Start with:

http://www.gstatic.com/s2/sitemaps/sitemap-000.txt

and increment 000 till you 404. From quick testing, it's less than 3500 but more than 3200.

View on HN

As of this comment it's exactly 3278. I wonder why you didn't bother doing 3-4 more trials before writing the comment :) http://www.gstatic.com/s2/sitemaps/sitemap-3278.txt

Because I spent too much time trying to figure out why the index sitemap XML file was returning a 404. I'm really surprised that Google, one of the inventors of sitemaps and big proponent of using them, can have such a high-profile error. I dug around to see if I could find what's going on.

By the time I got to figuring out how many text files there are, I was getting bored.

Reply on news.ycombinator.com