525 karma · joined November 20, 2012
posinsk () gmail.com
Now I'm thinking I could have a separate database file per "batch", store it in object storage and then download on demand and query it as I want. This way I'll not bloat my primary storage size as well as I dont need a special vector DB since sqlite vector search will be enough for up to 50k vectors.
Am i missing anything?
multiple processes connected to it.
Additionally you could experiment with a reranker instead of an LLM or after reranking take top-3 results and then feed to LLM as input in order to reduce input token costs.
Imagine you have a cheap and 99% accurate ocr. The other 1% you can detect and apply more powerful (more accurate but slower and more expensive) ocr method. What would you use? At scale these things add up.
If you’re interested you can find contact to me via this profile.
3.5 usd/1000 pages is just too expensive…
I wrote it before, OSS is changing in a way that will basically make it useless.
Now way I could have come up with Kimi Delta Attention.
Heads up, in 12 months from now, 90% of open source projects will be vibecoded, without actual users - only looking "interesting" (if not already?).
Its easy to vibe code a full blown compiler these days, the question is: will it be maintained? A catchy title is not enough to get sticky users, long term maintenance, community around the project is (or at least used to be).
Having said that, the only viable reason for these kind of projects I see is getting attention for a moment, boosting reach of Vercel logo across the web and then disappearing (or stop development).
If Vercel would think seriously about this project: they would show skin in the game - for example implement it as their own experimental runtime.
Worst is with LLMs and coding agents, I give it a task to work on then go browsing the web, the LLM finishes in 2 minutes but I come back after 10…
At the end of the day, my mind feels super tired even though its the LLM that did the heavy lifting. I also barely remember what I was browsing 15 minutes ago. At this point I can honestly say - Im in trouble.
I guess this is a universal method. In the time when we can have 100 pages vibecoded in an hour by agents, how do we decide which are actually worth human time?
The agent doesn't feel a notion of cost or time, for them producing slop is free. For us humans, reading slop is at cost of cognitive overload.
The only „for AI agent” solution that will gain adoption is/should be measured by two dimensions: token usage and correctness.
If the solution doesnt use less tokens than generating chart.js code for example - why should I use it?
Same for correctness, if the generated chart spec is correct only 90% of the time - why should I use it?
It’s still early but I think this is the direction we should be thinking about „for AI agents” libraries and projects.
I’ll tell you even more, in enterprise b2b saas, usually the company paying few thousand per month would have less questions and requests than the one paying few hundreds.