We provide a second opinion service with certified human radiologists, if anyone's interested: https://expert.med
289 karma · joined May 24, 2013
Remarkably, these figures and their writings, dating from ~2300 BC, were as distant from Julius Caesar as he is to us, and yet they played a major role in shaping our world, for instance by setting the early foundations for Judeo-Christian thinking (examples: the flood story, Enheduanna's laments, etc). So we have every reason to be interested in them.
It would, of course, be great to do both. But my point is that it is going to be much harder to attract funders, participants, press coverage, and so on for reading Mesopotamian tablets than for reading Greek or Roman papyri excavated from Piso’s villa in Herculaneum.
We have large volumes of clay tablets from Mesopotamia that pre-date these papyri and are considerably easier to read that get nowhere near the attention. E.g. the library of Ashurbanipal.
Several reasons are at play I suppose - the excitement and the drama are much higher with this. But I think the West's obsession with the Graeco-Roman world is also a major factor.
He worked on the Mona Lisa for 16 years, adding a brush stroke here and there until his death, never handing it to the wool merchant who commissioned it or his wife who was the subject of the painting.
His work is largely a collection of drafts and anti-*'s, but that hasn't taken away from his transformative role in the history of art, science, and engineering. There is beauty in unfinished work and in what we abandon. Finality is not necessary for greatness.
It's clear to me that the tech just isn't there yet. The information density of a web page with standard representations (DOM, screenshot, etc) is an order of magnitude lower than that of, say, a document or piece of code, which is where LLMs shine. So we either need much better web page representations, or much more capable models, for this to work robustly. Having LLMs book flights by interacting with the DOM is sort of like having them code a web app using assembly. Dia, Comet, Browser Use, Gemini, etc are all attacking this and have big incentives to crack it, so we should expect decent progress here.
A funny observation was that some models have been clearly fine tuned for web browsing tasks, as they have memorized specific selectors (e.g. "the selector for the search input in google search is `.gLFyf`").
[1] https://www.youtube.com/watch?v=tWwCK95X6go&ab_channel=Felix
Love the simplicity of this.
The key differentiator is privacy and local control. When users need to automate tasks on sites where they're already authenticated (banking, personal accounts, work systems), they need their actual browser with their existing sessions and cookies, not a remote instance.
You can use it to check and summarize news and social media, fill out forms, send messages, book holidays, do your online shopping, conduct research, and pretty much anything else that can be done within a browser.
The fact that Chrome and Gemini are, at least for now, owned by the same company raises huge privacy and consumer choice concerns for me though, and I see benefit in letting the user choose their model, where/how to store their data, etc.
Some of the cheaper models have very similar performance at a fraction of a cost, or indeed you could use a local model for "free".
The core issue though is that there's just more tokens to process in a web browsing task than many other tasks we commonly use LLMs for, including coding.
I might write a short post on the development process, but in short:
- started development during Easter so roughly a month so far
- developed mostly using Cline and Claude 3.7
- inspired and borrowed heavily by Cline, Playwright MCP, and Playwright CRX which had solved a lot of the heavy lifting already - in a sense this project is those 3 glued together
I don't plan to monetize it directly, but I've thought about an opt-in model for contributing useful memories to a central repository that other users might benefit from. My main aim with it is to promote open source AI tools.
Would love to explore a FF port. Right now, there are a couple of tight Chrome dependencies:
- CDP - mostly abstracted away by Playwright so perhaps not a big lift
- IndexedDB for storing memories and potentially other user data - not sure if there's a FF equivalent
I like that suggestion. Saved prompts seem like an obvious addition, and having templating within them makes sense. I wonder how well would "for each of the following websites do X" prompts work (so have the LLM do the enumeration rather than the client - my intuition is that it won't be as robust because of the long accumulated context)
Edit: forgot to mention it does support Ollama already
One can imagine an agent-to-agent marketplace where agents publish and consume such memories, standardized by canonical references to the MCP tools they've used, and possibly put a price on it based on how much work it would take another agent to "discover" that useful computational path. Then the consuming agents can make a "build vs buy" decision.
The core issue is in creating meaningful notions of "context" across the universe of tasks and environments. I'm skeptical of embeddings for that reason, and I think reducing false positives/negatives for cache hits is more important than efficiency in the short term, so perhaps a rich textual description of the context is a good short term compromise.
This is missing an important nuance. The valuable data generated through coding copilots isn't the code, but the step by step human-AI interaction to produce the code.
Windsurf and Cursor are effectively crowdsourced data annotation farms which give their owners an edge in providing better coding models. I think this is way more important than talent or software in this deal.
Sounds like a sensible next step would be to try and replicate this in animal models (i.e. treat animals with growth hormones extracted from cadavers of same species) to identify the proteins/prions that trigger the pathogenesis of Alzheimer's, which could perhaps be drug targets for at least a subset of AD causes.