I've been building scrapers and browser automation workflows for the past 10 years. I've learned that making something work once is easy but keeping tens of thousands of sessions working every day is not. There was a lot of trial and tribulation: reverse engineering APIs, forging requests, and stitching HTTP and browser automation together just to make it work.
I've packed all my knowledge into this new product im launching: Nina Technologies. We help AI agents use the web reliably (think Muse getting blocked/challenged on Amazon, credit card account site, etc), so other teams don't have to figure it all out the hard way.
Would love your feedback on the problem and solution. I'm happy to help anybody dealing with this kind of stuff, for free.
Cheers