Would you please describe the basics of this process? I am interested in doing this and my hardware is similarly limited. No dedicated GPU, but 24g ram and solid CPU.
https://github.com/Jeffser/Alpaca
GPT4ALL on the cinnamon mint box I first installed Easy Flatpak then installed it from there. That I had a little different goal, I had a lot of pdf and epub books I wanted to index. I downloaded a small model again, a 4.34G Llama 3 8B Instruct. In the local docs I put one pdf 'Github for Dummies' and asked it to index. It took about 30 minutes IIRC. (don't start with too many, it could be days indexing. You can add docs later and re-index so you can build your database slowly) Then instead of looking up in the book how to revert a commit, you chat with the model and ask it how to revert a commit.
My take is the model is the human language interface for queries, and that's all. Bear in mind almost everyone knows more about this me, I just wanted to see what all the fuss was about.
You can also do similar what I did with gpt4all online for free as long as you have a google account. Go to notebooklm dot google dot com and start uploading your documents.
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run llama3.2:1b
Then try bigger models if that is too fast for you :)