suchintan

The craziest one we heard about was this government portal in India that was hard to automate because halfway through the portal you had to refresh the page a bunch of times to get a button to show up

We thought it was inevitable that OpenAI / Anthropic would veer into this space and start to become competitive with us. We actually expected OpenAI to do it first!

What this confirms is that there is significant interest in computer / browser automation, and the problem is still unsolved. We will see whether the automation itself is an application later problem (our approach) or whether the model needs to be intertwined with the application (Anthropic's approach here)

We get a lot of requests for bad usage (ie spinning up upvote rings on Reddit) but we don't want to support things like that

This is another interesting example where it was allowed

We use residential proxy networks when running Skyvern to help simulate real human behaviour (because that's what Skyvern is trying to do).

We run headful browser instances (meaning a real chrome instance running with a real viewport) for the same cause!

We had an internal discussion about how to manage dependencies effectively, and we made the decision accept the risk that comes with blindly relying on Chrome for now, instead of investing heavily in mitigating that risk today.

The main motivator was for us to continue moving fast, and accept that we have a few hard dependencies in our business.

The goal is to find product market fit, then allocate time to de-risk some of these hard dependencies. If we fail to find product market fit, this may not matter at all

It's adapted from vimium and works like a charm. Distill the html down to it's important bits, and handle a ton of edge cases along the way haha

Interesting call about validating this against YAML / TOML / CSV / HCL. HTML was natural for us because we're dealing with web data, so the translation ended up being surprisingly intuitive

1. Do you intend Briefer to be a jupyter replacement or a metabase replacement?

2. Do you have any plans to open source briefer?

This is pretty much what we're building at Skyvern. The only problem is that inference cost is still a little bit too high for scraping, but we expect that to change in the next year

I've seen a lot of success come from AI sales agents, just doing basic SDR style work

We're having some success automating manual workflows for companies at Skyvern, but we've only begun to scratch the surface.

I suspect that this will play out a lot like the iPhone era -- first few years will be a lot of discovery and iteration, then things will kick into superdrive and you'll see major shifts in user behavior

Skyvern currently intercepts all network calls that gets made -- we save them all as a HAR file for debugging purposes... but never looks at them

A lot lot lot of scraping use-cases become simpler if you can just inspect a search api or a details API and get the information you're looking for. I'll add this to our roadmap!

For example, if Skyvern was asked to log-in to a website and do a search for product X, the generated action plan would include: 1. Click the log in button 2. Click "sign in with email" 3. Input the email address retrieved from source X 4. Input the password retrieved from source Y 5. Click log in 6. Click on the search bar 7. Input the search term from source Z 8. Click Search

Now, if the layout changed and suddenly the log-in button had a different XPath, you have two options: 1. Re-generate the entire action plan (or sub-action plan) 2. Re-generate the specific component that broke and assume everything else in the action plan still works

Now imagine it from the accountant POV, where they have the same use-case for hundreds of clients

This is where we've seen something like Skyvern really shine. It's targeting industries and companies that are doing rote work at a significant scale

Thank you for this feedback

We've been approaching it a little bit differently. We think larger more capable models would actually immediately improve the performance of Skyvern. For example, if you run it with LLaVa, the performance significantly degrades, likely because of the coupling

But since we use GPT-4V, and it's rumoured to be a MoE model, I wonder if there's implicit decoupling going on.

I'm gonna spend some more time thinking about this

Feel free to email me at suchintan@skyvern.com -- I can let you know when the self-serve UI is live

Here's a prompt example to try out

{ "url": "https://news.ycombinator.com", "navigation_goal": "goal is met if you see a post from basiep2. Terminate if you don't" }