2 is just a generalization of 1 and 3. To expand on 4, currently I display how many crawler requests were made and the average response time. If I were to further invest, I'd split these metrics by path requested and have it available over any time interval.
Do you have any ideas on what other pain points I could solve with this idea?