SlimerJS: a replacement for PhantomJS using recent Firefox
slimerjs.org
slimerjs.org
One of the most active browser automation libs out there and it's just a pure joy to use =)
Firefox did start investigating supporting the Chrome DevTools Protocol: https://groups.google.com/d/topic/mozilla.dev.platform/4-4A8...
We (Chrome DevTools/Puppeteer team) have had detailed discussions with their DevTools team on this, recommending what subset would enable Puppeteer. I don't know where they are with this, but in the long-term, this solution looks most promising.
Shameless fanboy moment: have followed your stuff for years, and I absolutely love the products your team(s) put out (CDP/Puppeteer)!
The devtools teams (Chrome, FF, Safari, and Edge) actually talk quite a bit. We collaborate on the console spec [0] and keep each-other in check on feature parity. ;)
Anyone from Mozilla know where to follow the work for this?
Personally, if I were to set up automated testing, I'd like to do it with both Chrome and FF, especially given that FF seems to be making a comeback lately. Another reason might be that I've never had anything but problems building Chromium from source, but have successfully built Firefox. So in situations where you want to know with certainty that your binary is derived from the published source code, Firefox wins IMO.
Chrome as a platform has the majority of browser market share and is continuing to be the top contender, I don't think anyone's going to knock them off their high horse anytime soon. Targeting the most common platform for applications deployment is common sense.
> especially given that FF seems to be making a comeback lately
FF is stagnant as far as market share, and has been trending down for at least 2-3 years. Currently it's around 5-6% and has been for awhile. Saying FF is making a comeback is disingenuous to the actual stats.
> Another reason might be that I've never had anything but problems building Chromium from source
Totally understand this one - I've found my roommate's Gentoo laptop in the fridge doing a full Chromium build more than a few times.
All-in-all I totally get where you're coming from, Chrome and FF are my preferred automation/testing platforms for sure =)
If I recall, people used to say something very similar about Internet Explorer. Of course Chrome isn't as awful as IE back then but still...
IMO it makes the most sense to support two independent implementations, especially for browsers.
>Currently it's around 5-6% and has been for awhile. Saying FF is making a comeback is disingenuous to the actual stats.
For desktop FF has about 10% marketshare. 5-6% includes iOS and Android, the later comes with a preinstalled chrome with little incentive to switch...
With the upcoming ESR 60 release, I also expect (and heard) a lot of companies will switch to Firefox, atleast in the german region (where marketshare increased from Dec to Jan). Some Linux distros might switch over from the previous ESR release and a lot of users could potentially reevaluate.
>I've found my roommate's Gentoo laptop in the fridge doing a full Chromium build more than a few times.
Chromium is not Chrome, it's Chrome with all the Google Binaries ripped out. Chrome itself is, to my knowledge, not open source, only Chromium is.
In the end, support chrome is simply supporting another Internet-Explorer-era. Maybe not as IE but it would be very similar.
If you or your company only develop for chrome you're not better than any of the websites that proudly state they will only work under IE6. End of story.
IMO - it is foolish to put Chrome and IE in the same box.
> If you or your company only develop for chrome you're not better than any of the websites that proudly state they will only work under IE6. End of story.
I don't think anyone is going to make an argument to only support Chrome, there's clearly other browser tech out there that we as devs must support. I will however say that you should prioritize supporting the most popular browsing platforms (browser, screen, device, OS, etc) when building applications and/or during testing.
Since you're touting Linux utilization - I will say that I better have a helluva testing budget (time and money) if I'm going to even touch on the long-tail platform combos (anything < 5%). The only reason my apps are heavily tested in Linux is because they're developed on a Debian Jessie box =D
What influence are you talking about? Can you be more specific?
For Web Testing spinning up the browser is okay, for some other tasks using just a headless version can be good enough.
I found browserless.io and the supporting github project over the weekend, looks promising.
Who else is doing this? Is the architecture I just touched on the right way to go about doing this? Or should I have each sidekiq job spin up its own browser container for the scraping job?
Also, chrome vs firefox won't really matter in this instance, but reliability/performance/memory will...any recommendations?
This is approximately how I was thinking of settings thing up as well, so thanks for this!
https://github.com/RealEstateWebTools/property_web_scraper
Will dig into your code to see what I can learn.
Chrome headless should be fastest. And if you're printing to pdf phantom and chrome had the better ability to do that..
I had each sidekiq job spinning up its own process -- and needed to do some X display avoidance.
https://docs.slimerjs.org/master/release-notes-1.0.html
"With Firefox 56 and more, SlimerJS can be trully headless by adding the –headless option on the command line"
So that could be an interesting thing to play with. I've thought about just having my sidekiq jobs spin up a docker container with the browser in it and then just limiting the amount of jobs at once for resource constraints, and now seeing your comment I think I might just give it a shot.