I would recommend using Headless Chrome along with a library like puppeteer[0]. You get the advantage of using a real browser with which you run pages' javascript, load custom extensions, etc.

[0]: https://github.com/GoogleChrome/puppeteer