fbouvier

Well bypassing anti-bot is not our focus. For us stable means having a stable web browser engine, ie. being able to handle webpages as close as possible as the specs and the traditional browsers like Chrome.

When I started this project more than 4 years ago (without LLM coding back then), I was confident I could build a browser from scratch in a systems language like Zig. The further I got, the more I realized what kind of madness I had gotten myself into, and dragged everyone who followed me into.

But in the end we made it: a working browser, used for billions of requests every day. Thanks to everyone who made this possible! Still a lot of work to do, obviously...

Keeping some kind of style information is definitely important to understand the semantics of the webpage.

Thanks Steeve!

Yes HTML is too heavy and too expensive for LLM. We are working on a text-based format more suitable for AI.

Thanks for the feedback, we will try to make this clearer on the website. Lightpanda works with Playwright, and we have some docs[1] and examples[2] available.

Web APIs and CDP specifications are huge, so this is still a work in progess. Many websites and scripts already work, while others do not, it really depends on the case. For example, on the CDP side, we are currently working on adding an Accessibility tree implentation.

[1] https://lightpanda.io/docs/quickstart/build-your-first-extra...

[2] https://github.com/lightpanda-io/demo/tree/main/playwright

Hi, I am Francis, founder of Lightpanda. We wrote a full article explaining why we choose Zig over Rust or C++, if you are interested: https://lightpanda.io/blog/posts/why-we-built-lightpanda-in-...

Our goal is to build a headless browser, rather than a general purpose browser like Servo or Chrome. It's already available if you would like to try it: https://lightpanda.io/docs/open-source/installation

And some lightweight alternatives like Bellard's QuickJS (https://bellard.org/quickjs/) in C and Kiesel (https://kiesel.dev/) in Zig.

Yes, argentic workflows are one of our use cases for Lightpanda.

We skip the graphical rendering of the web page for instant startup, fast execution and low resources usage.

It might works if you need to handle a few websites. But this retro engineering approach is not maintainable if you want to handle hundreds or thousands of websites.

I don't expect it to change a lot. All the main components are there, it's mainly a question of coverage now.

Not now but we might do it in the future. It's easy to export a Zig project as a C ABI library.

Thank you! Happy if you use it for your e2e tests in your servers, it's an open-source project!

Of course it's quite easy to spin a local instance of a headless browser for occasional use. But having a production platform is another story (monitoring, maintenance, security and isolation, scalability), so there are business use cases for a managed version.

It was my first idea. Forking Chromium has obvious advantages (compatibility). But it's not architectured for that. The renderer is everywhere. I'm not saying it's impossible, just that it did look more difficult to me than starting over.

And starting from scratch has other benefits. We own the codebase and thus it's easier for us to add new features like LLM integrations. Plus reducing binary size and startup time, mandatory for embedding it (as a WASM module or as C lib).

For now we just support CDP. But Selenium is definitely in our roadmap.

Yes but WebKit is not a browser per se, it's a rendering engine.

It's less resource-intensive than Chrome, but here we are talking orders of magnitude between Lightpanda and Chrome. If you are ~10x faster while using ~10x less RAM you are using ~100x less resources.

There are a lot of uses cases:

- LLM training (RAG, fine tuning)

- AI agents

- scraping

- SERP

- testing

- any kind of web automation basically

Bot protection of course might be a problem but it depends also on the volume of requests, IP, and other parameters.

AI agents will do more and more actions on behalf of humans in the future and I believe the bot protection mechanism will evolve to include them as legit.

Thanks! No we haven't worked on bot detection.

Every tool can be used in a good or bad way, Chrome, Firefox, cURL, etc. It's not the browser who doesn't play nicely, it's the user.

It's the user's responsibility to behave well, like in life :)

We did not run benchmarks with chrome-headless-shell (aka the old headless mode) but I guess that performance wise it's on the same scale as the new headless mode.

I fully understand your concern and agree that scrapers shouldn't be hurting web servers.

I don't think they are using our browser :)

But in my opinion, blocking a browser as such is not the right solution. In this case, it's the user who should be blocked, not the browser.

WASM support is not there yet but it's on the roadmap and we had it in our mind since the beginning of the project, and have made our dev choices accordingly.

So yes it could be used in a serverless platform like Cloudflare workers. Our startup time is a huge advantage here (20ms vs 600ms for Chrome headless in our local tests).

Regarding v8 in Cloudflare workers I think we can not used directly, ie. we still need to embed a JS engine in the wasm module.

We had some discussions about it. It seems to us that AGPL will ensure that a company running our browser in a cloud managed offer will have to keep its modifications open for the community.

We might be wrong, maybe AGPL will damage the project more than eg. Apache2. In that case we will reconsider our choice. It's always easier this way :)

Our underlying library https://github.com/lightpanda-io/zig-js-runtime is licensed with Apache2.

Yes, that's true. It's a balance to find between RAM and speed.

I was thinking more on use cases that require to disable JIT anyway (WASM, iOS integration, security).

In the past I've run hundreds of headful instances of Chrome in a server environment using Xvfb. It was not a pleasant experience :)

I guess it depends on the scale of your requests.

When you want to browse a few websites from time to time, a local headful browser might be a solution. But when you have thousands or millions of webpages, you need a server environment and a headless browser.

You're right, the debugging part is a good use case for graphical rendering in a headless environment.

I see it as a build time/runtime question. At build (dev) time I want to have a graphical response (debugging, computer vision, etc.). And then, when the script is ready, I can use Lightpanda at runtime as a lightweight alternative.

I believe it will be still significantly lower as we skip the graphical rendering.

But to validate that we need to increase our Web APIs coverage.

The cost of the browser part is still a problem. In our previous startup, we were scraping >20 millions of webpages per day, with thousands of instances of Chrome headless in parallel.

Regarding the RAM usage, it's still ~10x better than Chrome :) It seems to be coming mostly from v8, I guess that we could do better with a lightweight JS engine alternative.

The most important "feature" is to increase our Web APIs coverage :)

But of course we plan to add others features, including

- tight integration with LLM

- embed mode (as a C library and as a WASM module) so you can add a real browser to your project the same way you add libcurl