May be misunderstanding what you mean by “browser agents” but I’ve done some web scraping that had dynamic content and it was easy with a simple chrome driver / gecko driver + scraper crate in Rust