1,107 karma · joined May 25, 2009
The browser extension is really cool and it provides a needed tool for the agent to use. It used the extension to show the page that it updated in the task document (the task doc is great too). However it showed me a page and did it was done, when it was clearly not done and not what I asked for.
I was expecting weaker tooling and a better model. I got good tooling and a not very good model.
Maybe 3.1 will deliver?
It's really well done and digs into all the details on sodium-ion. Lots to like with sodium-ion (charge rate for one) but cost isn't going to be competitive for at least 5 and more likely 10 years.
One of my stumbling blocks with text diffusers is that ideally you wouldn’t treat the tokens as discrete but rather probably fields. Image diffusers have the natural property that a pixel is a continuous value. You can smoothly transition from one color to another. Not so with tokens. In this case they just do a full replacement. You can’t add noise to a token, you have to work in the embedding space. But how can you train embeddings directly? I found a bunch of different approaches that have been tried but they are all much more complicated than the image based diffusion process.
That is very impressive.
Side note: Superficially reminds me of Hierarchical Temporal Memory from Jeff Hawkins "On Intelligence". Although this doesn't have the sparsity aspect, its hierarchical and temporal aspects are related.
https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://www.numenta.com