HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by Dwedit | Hacker News Reader
Parent
Full thread
Dwedit
·
Recursive wget isn't going to pass Anubis.
View on HN
vaylian
·
This. Aggressive AI crawlers have ruined many parts of the web. I guess you have to ask the owner of the site nicely, if they would provide a complete copy of the site for your personal archive.
Reply on news.ycombinator.com