I chunked my conversations by day so that each conversation in the dataset would be about the same topic throughout without random switching, which isn't perfect, ideally I would let an LLM chunk the conversations into logical start/stop points, but I didn't want to spend all that money on tokens. I also got rid of any conversations with images and group chat conversations to simplify.
Not really. There are some, but I do think generally my conversations kinda biased the model over towards adjacent topics. I also included something in the system message along those lines ("Hi, I have a cute girl I want you to meet") for a sample response to a basic input like "Hi" just to make the conversation more interesting.