Hacker News Reader

Comment by jotto | Hacker News Reader

I've been successfully using Chrome headless in a 500MB Docker container for dumping the DOM for https://www.prerender.cloud/ for months (rendering a large variety of sites without restarts for weeks at a time)

Run it with:

  --js-flags="--max_old_space_size=500"

to force the VM to keep it GC'd below 500

Chrome v55 was a 30% memory savings, before that I used 1GB containers.

It's not perfect, but I am definitely pushing high volume (multiple tabs, concurrent activity) and I am not having any significant stability issues and I am pushing diverse sets of webpages.

View on HN

What the best strategy that worked for you on high volumes, multiple tabs or multiple docker instances? I am wondering if multiple tabs is as efficient as multiple windows/instances.

Thanks.

I'm using more than one Chrome process so I can kill the processes every so often (e.g. after timeout or when they get stuck). Inside each Chrome instance I use 16 tabs, there might be a number of factors at play:

- Are you worried about same-origin pollution if you run multiple tabs from the same origin in the same process? If so -> Extra process - Do you have to take screenshots? You can only take screenshots of the tab that's in the foreground, so you have to activate it first to take the screenshot. This might fail if you have lots of tabs which roughly trigger at once.

You can see what I've built at https://urlscan.io btw.

That ought actually be:

  --js-flags="--max-old-space-size=500"

Hyphens not underscores.

A list of V8 flags can be found with:

  --js-flags="--help"

I think it may accept both the underscore and hyphen and just normalizes it.

This is correct, it does accept both, and the help lists it as underscores not hyphens. So the comment about it ought to be hyphens not underscores is incorrect. Either is fine.

When I went digging through the code for v8 I believe I saw examples of it both ways. I'm not dead sure though because I don't know very much C. I just checked because I was hoping they weren't actually using an invalid flag for all that time.

what Docker image are you using?

Reply on news.ycombinator.com