https://github.com/webrecorder/browsertrix-crawler works pretty well. Same tech as https://archiveweb.page/ but non-interactive