This is a great idea.
I want to make a 'pet' for my kid.
I can't get them a real dog, so why not a tinyLLM?
Training on guttenberg data is a great idea. What I would do is train it on all the e-books I have that are suitable for kids (I managed to find quite a lot online).
The dad jokes idea is great, please keep doing things along this line.