Hacker News Reader

Comment by vitovito | Hacker News Reader

Note: not genuinely headless. It still requires you run an X server, such as Xvfb.

As far as I know, there's no genuine headless Webkit.

For a genuinely headless system, consider Chris Lord's "offscreen" branch of Gecko, discussed here: http://blog.mozilla.com/ted/2010/07/29/moz-headless-screensh...

I'd be interested in sponsoring someone who wanted to bring the offscreen branch up to mainline, and Chris would be willing to mentor you in such an effort.

View on HN

Yeah, I wanted something like this a couple years ago and was disappointed to find no true headless library. I spent a lot of time looking at WebCore, but eventually gave up after a while.

I wonder if there's a way to extricate some of Chrome's rendering logic to do genuinely headless stuff. I know that things are rendered in separate processes and then piped to the browser window process and I have to wonder if there's a way to launch one without the other, and thus, without any requirements for a display server like X.

If you're okay with using Gecko instead of Webkit, the headless branch does compile into a standalone executable you can fork out to, it's what we do.

The issue with Gecko was less about rendering offscreen and more about rendering widgets (dropdowns, input boxes, etc.) which are normally handled by the windowing system.

EA's branches of WebKit, available at http://gpl.ea.com, are designed for integration into console video games and don't require any sort of X server or OS interface. It's not the most up-to-date but it's sufficiently modularized to adhere to the LGPL without violating the Sony/Microsoft developer agreements.

This might be interesting. Remember to check all of the products listed, as we only release our latest EA WebKit version when a game ships with it.

"EA WebKit supports Win32, Win64, Playstation 3, and Xbox 360 platforms" according to the documentation, but perhaps the modularization and the EA Raster primitives gets you further along than some of the other forks.

If anyone has any questions about EA WebKit, let me know and I can pass them along.

EDIT: ...Or you can ask the parent poster, who actually works on them. :)

I use htmlunit for scraping, it's a headless browser although using WebKit will be far better.

It's sad that WebKit lacks some easier integration (good COM/.NET object in Windows).

Crowbar is a Gecko-based scraper, if you're interested in using a real browser: http://simile.mit.edu/wiki/Crowbar

The issue is using Crowbar in latest Gecko versions.

I've used htmlunit as well, basically because it seems to be the most mature. I find it chokes on some google maps js.

We created a genuinely windowless branch of Chromium two years ago named Awesomium; it's free for non-commercial use and is used in several major games and commercial projects: http://www.awesomium.com

"Runs on Windows and Mac OSX Intel platforms."

If you only run on Windows or OS X, it's windowless, but it's not headless.

Headless means something running on a command-line console, and you can't run either of those operating systems that way.

We're working on a Linux port for use server-side but that's not to say our current framework is not headless. We developed this branch to be completely independent of any windowing systems (and have even coaxed Flash and Silverlight to render correctly in such an environment)-- check out our HelloAwesomium sample in the SDK, it runs straight from the command-line and outputs a JPEG.

Reply on news.ycombinator.com