Storm: LLM system that researches a topic and generates full-length wiki article
github.com
github.com
Publishing AI generated material is generally speaking a horrible idea and does nobody any good (at least until accuracy levels get much much better.)
Even if they do it well and truthfully (which they don't) current LLMs can only summarize, digest, and restate. There is no non-transient value add. LLMs may have a place to help query, but there is no reason to publish LLM regurgitations alongside the ground truth used to generate them.
Though, at a stretch, Wikipedia itself could be considered based around summarization, digesting, and restating/citing things said elsewhere, given its policy of verifiability: "Even if you are sure something is true, it must have been previously published in a reliable source before you can add it." Now, LLMs aren't well known for their citation skills, to be fair.. :-)
Though note that there still isn't any need to publish static content. The power of LLMs is that they can be dynamic and responsive!
Even if we hypothesize that it were possible for a LLM to write a high-quality wikipedia-like output, generating the whole thing statically in advance like existing Wikipedia would be relatively pointless. It'd be much more interesting to generate arbitrary (and infinite!) pages on demand.
The hardest part of good documentation is getting started. Once there are docs in place it's usually much easier to revise and correct than it would have been to write correctly by hand the first time. Think of it like automating a rough draft.
Sometimes, the output the LLM generates is correct and I'm just approving it. Other times, it is mostly right, and I can easily identify and correct its errors. Yet other times, it is totally wrong, but often typing out why it is wrong is a good start to actually generating correct text. Often (but not always), the kinds of false assumptions which LLMs make are similar to those a human reader would make, so stuff like "A and B sound very similar but, in the context of this system, actually have completely different meanings" is a useful addition to documentation anyway.
The worst case scenario is the LLM generates something which is subtlety wrong, and the human review fails to pick up on the subtle error. But, that's something which can happen even with no LLMs involved at all. It isn't uncommon for people to make subtle errors in documents they write (often because they are misremembering something) and for those subtle errors not to be picked up during the review process. I'm not convinced the odds of this happening with an LLM-assisted workflow are significantly greater than with a purely human workflow.
No, you're wrong. LLMs create new experiences after deployment, either by assisting humans, or by solving tasks they can validate, such as code or game play. In fact any deployed LLM gets to be embedded in a larger system - a chat room, a code running environment, a game, a simulation, a robot or inside a company - it can learn from iterative tasks because each following iteration carries some kind of real world feedback.
Besides that, LLMs trivially learn new concepts and even new skills with a short explanation or demonstration, they can be pulled out of their training distribution and collect experiences doing new things. If OpenAI has 100M users and they consume 10K tokens/user/month, that makes for 1 trillion tokens of human-AI interaction rich with new experiences and feedback.
In the text modality LLMs have consumed most of the high quality human text, that is why all SOTA models are roughly on par, they trained on the same data. That means easy time is over, AI has caught up with all human language data. But from now on AI models need to create experiences of their own, because learning from your own mistakes is much faster. The more they get used, the more feedback and new information they collect. The environment is the teacher, not everything is written in books.
And all that text - the trillions of tokens they are going to speak to us - in turn contributes to scientific discoveries and progress, and percolate back into the next training set. LLMs have massive impact at language level on people so by extension on the physical world and culture. They have already influenced language and the arts.
LLMs can create new experiences, learn new skills, and have a significant impact through widespread deployment and interaction. There is "value add" if you look at the grand picture.
Yeah this is really having a positive impact on scientific discovery.
Here is a LLM with search doing competitive level coding:
https://deepmind.google/discover/blog/competitive-programmin...
and in general, applying evolutionary methods on top:
https://scholar.google.com/scholar?cites=1264402983985539857...
The explanation is simple - either learn from past experience which is human text for now, or learn from present time experience which comes from the environment. The environment is where LLMs can do novel things.
A system like this is multilayered, with prompts going through the whole problem solving process, considering the information presented, assuring quality and factuality, assigning the necessary citations and documentation for claims.
Accuracy isn't a problem. The way in which AI is used creates the problem - ChatGPT and most chat based models are single pass, query/response type interactions with models. Sometimes you get a second pass with a moderation system, doing a review to ensure offensive or illegal things get filtered out. Without any additional testing and prompt engineering, you're going to run into hallucinations, inefficient formulations, random "technically correct but not very useful" generations, and so forth. Raw ChatGPT content shouldn't be published without significant editing and going through the same quality review process any human written text should go through.
What Storm accomplishes is an algorithmic and methodical series of problem solving steps, each of which can be tested and verified and validated. This is synthesized in a particular way, intended as a factual reference article. Presumably you could insert debiasing and checks for narrative or political statements, ensuring attribution and citation occur for quotations, and rephrasing anything generated by the AI as a neutral, academic statement of fact with no stylistic and artistic features.
This is significantly different from the almost superficial interactions you get with chatbots, unless you specifically engineer your prompts and cycle through similar problem solving methods.
Tasks like this are well within the value add domain of current AI capabilities.
Compared to the absolute trash of SEO optimized blog posts, the agenda driven, ulterior laden rants and rambles in social media, and the "I'm oh-so-cleverly influencing the narrative" articles posted to Wikipedia by humans, content like this is a clear winner in quality, in my opinion.
AI isn't at the point where it's going to spit out well grounded novel answers to things like "what's the cure for cancer?" but it can absolutely produce a principled and legible explanation of a phenomenon or collection of facts about a thing.
I think the opposite. Publishing material generated like this is going to dilute our knowledge base and result in worse content.
But we're both speculating. Only time will tell. I hope that you're right and I'm wrong.
This moment reminds me very much of the late 1990s when it was common knowledge that "claim jumping a domain name" is very rude and not advisable, or the common knowledge among intellectuals that "ads will ruin the Internet" .. yes, polite people did not make companies to claim jump domain name registration, or push annoying and repetitive ads on the Internet..
but..
If it doesn’t have value for being saved and published, why would it have value for the person viewing it ephemerally?
One reason to NOT want to persist an LLM output is to avoid contaminating the internet with LLM text that could be confused for human text. This is useful for historical reasons (what did people say/think) and for future LLM training purposes.
In the future, there might be other reasons to not persist LLM-generated text. If you store the inputs instead of outputs then it can be "replayed" with a new LLM. I can generate some text via an LLM running on my phone, but if I later wanted a higher-quality output, it'd be easier to re-generate the data on my laptop later (or the cloud, obviously). One related idea that comes to mind is personalization - let the LLM use language each reader is familiar with, or focusing on different bits of info to different people. Maybe if an LLM knows I have a PhD in a certain subject, it'd summarize a research paper differently than it'd summarize it for my neighbor who has different preexisting knowledge.
There are wikipedias in other languages - Maybe this framework could be adapted to translate the search terms, fetch mulitlingual sources, translate them back, and use those as comparisons.
I've found a lot of stuff out through similar by-hand techniques that would be difficult to discover on english search. I'd be curious to see how much differential there is between accounts across language barriers.
For broader implementation you would want to develop the approach further. The idea of sampling other-language Wikipedia mentioned in a sibling comment seems to be a decent next step.
Extending it to bringing in from wider sources would be another step. I doubt it would be infallible but it would be really interesting to see how it compares to humans performing the same task. Especially if there were a additional ability to verify written articles and make corrections.
If your goal is to generate a wiki article, you can't assume one already exists. That's begging the question. If you could just search wikipedia for the answer, you wouldn't need to generate an article.
Otherwise it would be like reading HN or Reddit debates where 2 egomaniacs who are both wrong continually straw man each other with statements peppered with lies and parroted disinfo, aint got time for that.
> While the system cannot produce publication-ready articles that often require a significant number of edits, experienced Wikipedia editors have found it helpful in their pre-writing stage.
So it can't produce articles that require many edits? Meaning it can produce publication-ready articles that don't need lots of edits? Or it can't produce publication-ready articles, and the articles produced require lots of edits? I can't make sense of this statement.
An AI assistant app that mixes AI features with traditional personal productivity. The AI can work in the background to answer multiple chats, handle tasks, and stream/feed entries.
https://old.reddit.com/r/LocalLLaMA/comments/1b8uvpw/does_fr...
Imagine an infinite wiki where articles are generated on the fly (from reputable sources - with links), including links to other articles (which are also generated) etc.
I actually like this sort of interface more than chat.
I mean, the spelling and syntax of the sentences is mostly correct, just like any LLM content. But there's ultimately still no coherence to the output.
What do the authors call what they're doing? Magic?
This is a hard and tmk unsolved NLP/IR problem, and data access is an issue.
> Second opportunity is fitting in things only found in other language Wikis that either be incorporated into the english one or help introduce new articles.
This has been attempted via machine translation in the past, and it failed because you need native speakers to verify and correct the translations and this wasn't the sort of work that people were jumping to volunteer to do.
For multilingual LLM’s? Why do you think that?
An LLM can translate inputs of arbitrary Russian text. If there were an English question about something only in the training data as Russian, I would expect an answer - with the quality being on par with its general translation capabilities.
There's a concurrent explosion of 'veracity' analysis - it'll be fun to run those against wikipedia a year from now and your data.
Incidentally, are you interested in mirroring your dataset and making it more robust? I'm sure I've got a few TB of storage lying around somewhere...
There's a small ironically funny typo in the first line: knolwedge
Wreaking havoc on the digital Akashic records.
Good luck with this storm, wiki's the world over. Just a thought but ... maybe someone should ask an org like the Internet Archive to snap-shot Wikipedia asap and label it Pre-Storm and After-Storm
what if that is not the case? what if the quality of this type of content actually increases?
And even if we accept the premise (as flawed as it might be) that Ai is not able to create original knowledge, most of what's online is dessimination and dies not represent new information but just old information rewritten to be understandable by a certain segment.
something LLMs excel at.
The current state of LLMs do hallucinate though. It's just not a very trustworthy source of facts.
LLMs was popularized less than 2 years ago.
I think it is safe to assume that it will be as trustworthy as you see Wikipedia today, and probably even more as you can embed reasoning techniques into the LLMs to correct misunderstandings.
Wikipedia cannot self correct.
unless you see Wikipedia as the organisation and not the encyklopedia?
in that case: sigh, then everything self corrects
Wikipedia is open, like completely open.
GPT is not.
Unless we manage to crack the distributed training / incremental improvement barriers, LLMs are a lot more likely to follow the Google path (that is, start awesome and gradually enshittify as capitalist concerns pollute the decision matrix) than they are the Wikipedia path (gradual improvement as more eyes and minds work to improve them).
it also carves I to the question what constituted model openness?
most people agree that just releasing weights are not enough.
but I don't think it will ever be feasible to say that reproducing model training is feasible. especially when factoring in branching and merging of models.
for me this is an open and super interesting question.
A model that can be incrementally trained (this is the bit we're missing) hosted by a nonprofit, belonging to "we the people" (like wikipedia).
The training process could be done a little like wikipedia talk pages are now - datasets are proposed and discussed out in the open and once generally approved, trained into the model.
Because training currently involves backpropagation, this isn't possible. Hinton was working on a structure called "forward-forward" that would have overcome this (if it worked) before he decided humanity couldn't be trusted [1]. It is my hope that someone smarter than me picks up this thread of research - although in the spirit of personal responsibility I've started picking up my old math books to try and get to a point where I grok the implementation enough to experiment myself (I'm not super confident I'm gonna get there but you can't win if you don't play, right?)
It's hard to tell when (if?) we're ever going to have this - if it does happen, it'll be because a lot of people do a lot of really smart unpaid work (after seeing OpenAI do what it did, I don't have a ton of faith that even non-profit orgs have the will or the structure to pull it off. Please prove me wrong.)
congrats on that!
On the other, moderation is an unsolved problem. The general mess of the internet is probably not quite ready to be handed a footgun of this caliber.
As with many things tech, some of the outcome falls to us, the techies. We can build systems to help steer this.
I think the real reason is one line dogmas like this.
To be clear - I'm with you that these systems can absolutely be a force for vast good (at least, I think that was what you were getting at unless there was a missing '/s'). I use them daily to pretty astounding effect.
I'll admit to being a little put off by being labeled dogmatic - it's not something I consider myself to be.
However, I do see a lot of one-sentence "truthms" being thrown around. like "garbage in; garbage out" and the likest.
these are not correct. we can just look at the current state of the art with LLMs that has vast amounts of garbage going in - it seems like the value is in the vastness of the data over the quality.
> On the one hand, a tool is as good or bad as the person wielding it.
I see this as being a dogme. smart people make good LLMs dumb people do not. but this is an open question. it seems like the biggest wallet will be the winner of the LLM game.
please correct me if I misunderstood something.
There are most definitely good and bad tools, in terms of more or less effective. Machine learning models are for sure outclassing a whole swath of tools in a number of domains and will more than likely continue to overtake purposes over time.
Whether this is a good thing for society is what I thought we were questioning - which is what I meant by steering. We can build tooling to do things like establish veracity, enable interrogation of models, and provide reasoning about internals (which we should do).
Open sourcing as much of this effort as possible will further lead to Good Things (because why are people working on these for free if they're not creating something of actual use) - whilst leaving all ML development to large corporations will inevitably ensure that the only thing you can trust an ML model to do would be spy on you and try to get you to buy stuff, because money.
God the grammar in that last sentence was terrible. But I think you get the point.
In another context, you may know this as an "echo chamber". Not quite exactly the same concept, but very, very similar.
I do like to remind people that the AI of today and LLMs are not the whole of reality. Perhaps someday there will be AIs that are also capable of directly consulting the universe, through some sort of body they can use. But the current LLMs, which are trained on some sort of human output, need to exclude AI-generated input or they too will converge on some sort of degenerate attractor.
currently we have mostly LLMs in the mix. but there are no reason that the Ai mix will not contain embodied agents thst also publish stuff in the internet. (think search and rescue bots that automatically write a report).
Now Ai is connected to reality without people in the mix.
Most model collapse studies explore degenerate cases to determine the potential limits of the training process of the same model. No wonder you will get terrible results if you recursively recompress a JPEG 100 times! In real world it's nowhere near that bad, because models are never trained on their output alone and always guaranteed to receive the certain amount of external data, starting from the manual dataset curation (yes, that's also fresh data in itself).
Meanwhile, synthetic datasets are entirely common. I suspect this is a non-issue that is way overblown by people misinterpreting these studies.
However, if AIs become as successful as Nvidia stock price implies, it could indeed become difficult to find text that is guaranteed to not be AI. It is conceivable that in 20 years it will be very difficult to generate a training set at any scale that isn't 90% already touched by AIs.
Of course, it's conceivable that in 20 years we'll have AIs that don't need the equivalent of millennia of training to come up to their full potential. The problem is much more tractable if one merely needs to produce megabytes of training data to obtain a decent understanding of English rather than many gigabytes.
Only fine-tuned models are producing impressive work, because when we say something is impressive it by definition means not like the status quo - the model must be tuned toward some bias or other, whether it's aesthetic or otherwise, in order to stand out from the rest. And generic models like GPT or Stable Diffusion will always be generic, they won't have a bias toward certain truths - they'll be mostly unbiased which we want for general research or internet search.
So it's interesting, in order to get incredible quality of work out of AI, you have to make it specific, but in order to that, you have to train it on the work of humans. I think for this reason AI will always be ultimately behind humans, though it of course will displace a lot of work we do, which is significant.
At the most generous level of scrutiny, the only part that could be related to research would be the aggregation of sources - but that is only a precursor to research and likely is too generalized to be as accurate as a specialist preparing data for actual research.
Ex: "the gyroscope is trying to stay upright", or "the computer complains because the update is broken" or "evolution will give the birds longer beaks".
That said, I agree that the problem is dramatically more-severe when it comes to "AI".
So many people think LLM means chatbot, even here on HN. So many people think agent means mentally humanoid.
But we have others, like Stable Diffusion's Web UI and Leonardo.AI - these are just tools with interfaces and the text entry for prompting is not presented as though it's a conversation between 2 people.
Someone shared an AI songmaker here recently... And there's a number of promising RAG tools for improving workflows for: Doctors, mechanics, researchers, lawyers.
I agree with you and expect the "AI character" use case to narrow significantly.
The presence of the word "scientific" in this statement disturbs me.