for that level of magic.. just use https://VisualSitemaps.com ;-) and yes, we also crawl password-protected sites, great for user-dashboards and staging.
hey there, i tested your service last year, does what is says greatly. Price point is not there for me sadly
“Please note, that we don't crawl website content and structure, we use only the public sitemap.xml file.”