Hacker News Reader

Comment by tnolet | Hacker News Reader

OP here with some background info. Headless Recorder started life as “Puppeteer Recorder” which was posted here earlier. However, we just added Playwright support and renamed the project!

View on HN

Looks amazing, it is generally very difficult to find well-maintained, open source web scraping/robot process automation tools. Thank you for building this and congratulations on the successful execution!

Do you ever run into issues with puppeteer scripts where you do waitUntil: networkIdle2 and every 10-15 page loads it just sits there forever? There's a timeout parameter that you can pass too, but it seems to do nothing.

I seem to have this problem often and I'm wondering how common it is.

I remember this problem. Different framework though. It’s probably something buggy deep in whatever driver library we are still using. Our solution was to not use that function.

I found that I need to listen for error events inside the page, particularly http 401 and 400 and abort if these come up

I'd be curious as to why change from an easily remembered and branded name to a generic one?

Edit: not bashing, genuinely curious

It used to support just generating Puppeteer scripts. They added support to also generate Playwright scripts.

Sticking with the old name would make it sound like they only supported Puppeteer.

This is correct. We want to do more than just Puppeteer, so it made sense.

Reply on news.ycombinator.com