This is so true - my favorite is when I have hard time to buy something and after I buy my dream thing, the flood of adds with better deals kill my happiness for getting it cheaply:-)
99 karma · joined June 8, 2018
For pretty long time I was thinking about making the application which shows AI leaders predictions vs. reality.
Like "We will have AGI in ..." or "There will be no SWE jobs in..." as I feel really bad when some of these "visionaries" influence decisions of young people planning their future, normal people decisions (and markets obviously).
Another chapter would be fact/reality check for statements like "We don't hire SWEs anymore..." while having 100s of opened positions on their job portal.
Anyway, great app!
I also completely understand "bloat"/simplicity - my project may seem dead on Github, but for me it is "perfect software" - no more features needed, no bugs in what I use daily.
Some of my favorite projects got so complicated over the years that I stopped to use them. This is why Don and his TeX is the legend ;)
I went on sabbatical to fulfill my dream project - consolidating years of training logs. I'm enjoying the technical challenges involved - digitizing paper hand written logs / visual models, navigating the maze of athletic metrics with their crazy trademarked names and multidimensional models. Having fun building AI coaches: agents ranging in character from Al Pacino in Any Given Sunday to the coach from my teenage years, utilizing ICL model-based predictions, ... and more.
The best part is the rush of memories while ingesting my own history - photos and recordings I completely forgot, as well as navigating data shared by friends.
This month has Strava & LeCol everesting challenge - signed up and added support that suits my needs to MyTraL. Good times.
I went on sabbatical to fulfill my dream project - consolidating 30 years of training logs that span everything from paper and Excel spreadsheets to various fitness services and devices I used. I'm enjoying the technical challenges involved - digitizing paper hand written logs using OCR / visual generative models, navigating the maze of athletic metrics with their crazy trademarked names and SOTA multidimensional models. Having incredible fun building AI coaches: agents ranging in character from Al Pacino in Any Given Sunday to the coach from my teenage years, utilizing ICL / PFN model-based predictions, ... and more.
The best part is the rush of memories while ingesting my own history - photos and recordings I completely forgot, as well as navigating data shared by friends - records they didn't see in years because the original applications they used no longer exist or won't run on their current HW.
I have been working on my note-taking application (https://github.com/dvorka/mindforger) for some time and wanted to go in the same direction. However, I gave up (for now). I used ggerganov/llama.cpp to host LLM models locally on a CPU-only machine with 32GB RAM, and used them for both RAG and note-taking use cases (like https://www.mindforger.com/index-200.html#llm). However, it did not work well for me - the performance was poor (high hardware utilization, long response times, failures, and crashes) and the actual responses were rarely useful (off-topic and impractical responses, hallucinations). I tried llama-2 7B with 4b quantization and a couple of similar models. Although I'm not happy about it, I switched to an online commercial LLM because it performs really well in terms of response quality, speed, and affordability. I frequently use the integrated LLM in my note-taking app as it can be used for many things.
Anyway, Reor "only" uses the locally hosted LLM in the generation phase of the RAG, which is a nicely constraint use case. I believe that a really lightweight LLM - I'm thinking about a tiny base model fine-tuned for summarization - could be the way to go (fast, non-hallucinating). I'm really curious to know if you have any suggestions or if you will have any in the future!
As for the vector DB, considering the resource-related problems I mentioned earlier, I was thinking about something similar to facebookresearch/faiss, which, unlike LanceDB, is not a fully-fledged vector DB. Have you made any experiments with similarity search projects or vector DBs? I would be interested in the trade-offs similar to small/large/hosted LLMs.
Overall, I think that both RAG with my personal notes as a corpus and a locally hosted generic purpose LLM for the use cases I mentioned above can take personal note-taking apps to a new level. This is the way! ;)
Good luck with your project!
hstr --show-configuration >> ~/.inputrc && . ~/.inputrc