One blog post how we built agent sandbox infrastructure with unikraft - we still use unikraft for agent sandboxes and love the technology.
160 karma · joined October 10, 2023
I grew up in a small village in Germany. 500 people, 5000 cows. Only farmers and a cheese factory. In the factory, we worked on Christmas, Easter, and New Year's Eve every morning at 5 am. Farmers don't take days off because cows don't take days off.
Maybe it's not the most healthy way of life. I don't think it physically requires us to take time.
Agent call 1: Send WhatsApp message (to=Magnus, text=hi) Inside, you open WhatsApp and search for Magnus (without LLM)
Agent call 2: Select contact from all possible Magnus contacts Script 3: Type the message and click send
So in total, 2 calls - with Gemini, you could already achieve this in 10-15 seconds.
Furthermore, valid point: if Pepsi spends $1M on ads, why don't you get a piece of it if they pitch to you?
Then we present this list to the LLM with the task and the LLM outputs input_text(id 3, Hello World).
Finally, we execute the Playwright code to perform the actual action of inputting text into this element.
The cool thing is that we can extract xPaths from the agent runs and re-run these scripts deterministically. I think that's a big advantage over pure vision-based systems like Operator.
A bigger problem on LinkedIn for us is all the nested UI elements and different scrolling elements. With some configuration in our extraction layer in buildDomTree.js and some custom actions, I believe someone could build a really cool LinkedIn agent.
On most platforms, browser use only requires the interactive elements, which we extract, and does not need images or videos. We have not yet implemented this optimization, but it will reduce costs for both parties.
Our goal is to abstract backend functionality from webpages. We could cache this, and only update the cache if eTags change.
Websites that really don't want us will come up with audio captchas and new creative methods.
Agents are different from bots. Agents are intended as a direct user clone and could also bring revenue to websites.
I am the creator of browser-use and build it with the vision that most websites will take a long time to adopt.
One vision behind us is to predict from a website directly higher abstract functions which the agent only needs to call and we execute code to process that.
One interesting thing could be to write into the html directly API descriptions which the agent can you simultaneously.
We definitely need a new dataset with more complex tasks, like uploading files, handling multiple tabs, and handling many more steps.