ArchiveBox Is Super Cool
mtlynch.io
mtlynch.io
It archives. A few hundred URLs. At least some of the archival methods work, some fail a lot.
And then it always stops. I‘ve never seen it archive a few thousand URLs, despite having it run for several days.
Zero error message, it just stops and seemingly idles,
Would be helpful to know the install method, some dependencies have memory leaks or child process killing issues that have caused stalling in the past. We have it running on demo.archivebox.io with tens of thousands of URLs and I haven't had to restart it in months, but different host systems have a lot of variation in stability still.
In the meantime this is a stopgap measure to fix the stalling problem.
archivebox schedule --every=day --update
This will make it pull missing methods once per day.It was the docker composer method.
If you feel inclined to try it again in a few months I guarantee we'll have changes pushed to improve stability. I'm working on it full-time these days so progress is more rapid than it's been in the past.