I still haven't seen strong evidence that fine-tuning to add extra knowledge is effective, but I'd be delighted to learn otherwise.
For example, could there be a site like HN with ten thousand contributors where the contributions are changes to an LLM rather than posts and comments?
One issue is that if contribution A contradicts contribution B, then on HN the contradiction presents no problem (i.e., two HN comments can and often do contradict each other just fine) whereas AFAICT the LLM will need to resolve the contradiction somehow to give coherent answers on the subject matter of the contributions A and B. Then again I suppose the LLM's answer could take the form, "opinions on [subject] vary, with some maintaining that . . . whereas others claim that . . ."
This is sometimes called RAG, for Retrieval Augmented Generation.
These days the most convincing way to do this is via tool calls.
Provide your LLM harness with a tool for running searches, and tell it to use that tool any time it needs additional information.
A good "reasoning" LLM like GPT-5 or Claude 4 can even handle contradictory pieces of information - they can run additional searches if they get back confusing results and work towards a resolution, or present "both sides" to the user if they were unable to figure it out themselves.
Let's say, just in time for Jesus to save you.
However, the frontier models keep improving at a quick enough rate that it's often more effective just to wait for the general solution to catch up with your task then to spend months training a model yourself. Unless you need a particular tightly controlled behavior or need a smaller faster model or what have you. Training new knowledge in can get weird [2].
And in-context learning takes literal seconds-to-minutes of time if your information fits in the context window, so it's a lot faster to go that route if you can.