"Codebases, docs, large corpses of internal datasets"

I still haven't seen a convincing demo of using fine-tuning to "teach" a model new information from additional documents. I'd love to see one.

(Closest I've come to that is I heard a rumor that Jane Street have fine-tuned an LLM for OCaml)