Hacker News Reader

Comment by jsheard | Hacker News Reader

This is what happens if your robot isn't nice

  > curl -I -H "User-Agent: Googlebot" https://www.cloudflare.com
  HTTP/2 403

That's not from robots.txt, but their Bot Management feature which blocks things calling themselves Googlebot that don't come from known Google IPs.

Are GCP IPs considered Google IPs?

No I am very sure they are not.

No.