Show HN: I trained a 65B LLM on my texts to talk to myself (details inside)
airic.serveo.net
I posted this to Reddit and had several hundred people talking to it. Salient points from that discussion:
LLAMA 1 65b
Rank 128
5 epochs
Batch size 1, 256 cutoff
Trained in the Oobabooga suite using bitsandbytes 4-bit quantization for the lora
Loss around 1.5 seems to give the most coherent results
Trained on raw text dumps that is then parsed by a crappy Blazor Server app I threw together in a few hours. Text format is just "Sender:The Message\n"
Trained on 2x 3090
Training took about 16 hours at a 90% power cap on the 3090's
Trained on ~30k texts (I talk a lot, that was just 2 years)
There's nothing telling it that it's a robot, though it sometimes seems to know
It was largely inspired by the Unreal Engine lora tutorial
I generated a list of fake names and addresses, pulled a list of my contacts, and then scripted out swapping the names and addresses for fictitious PII. I don't really send other sensitive data through text and my account is so thoroughly associated with my real name/location that the data leakage risk is manageable for the short period of time I'll have this available. It tends to halucinate fake PII as well which I think is partially a side effect of the data scrubbing. You'll notice it says things like that I live at 420 Ligma.
I'll need to mix in some actual assistant tasks to the dataset before it will actually be useful as an assistant. Right now it's largely just for idle conversation.
It's pretty ADHD and will randomly go off on its own tangents. I don't think it's the model. I think I just talk like that.
Let me know if you have any questions or comments. I built it for myself, but figured I'll let the communities that have taught and entertained me so much play with it a little, too.
Note: it says some pretty unhinged stuff. There's absolutely no guardrails. It also tends to talk like you're already friends with history.