Something about all this feels a bit backwards though, ultimately all these websites have some underlying API they're hitting. The "right" thing would be to just expose that API directly, and then an LLM could convert user input into a request that hits the endpoint. I think that was supposed to be the vision of Siri and such, but it never materialized because even if 3rd party integration was feasible natural understanding wasn't cracked yet.
Now that we finally have good natural language understanding, instead of executing on that vision, we're going backwards and having agents navigate websites as humans do?? If it does take off, the natural end-point is going to be that pages might have an alternate version that's simplified to make it easier for such agents to do their thing, just as web pages adapted for mobile formats.