GPT is already capable of incredible generalized language understanding and I'd wager we've long since hit diminishing returns from raw internet data. RLHF, fine tuning, and better (and more data efficient) architectures are what we need now.
GPT is already capable of incredible generalized language understanding and I'd wager we've long since hit diminishing returns from raw internet data. RLHF, fine tuning, and better (and more data efficient) architectures are what we need now.
I'm sure you can do lots of interesting research using outdated datasets but for companies creating products this will not be sufficient.
If your data or models don't account for those then it can make mistakes. For example, if a model is only trained on modern sources (and does not know about Early Modern English 2nd person pronouns "thy"/"thine"/etc.) then it can easily get confused when determining parts of speech, which then affects other down-stream processing.
Would help to stay up to date on current events, and up to date on its general understanding of the world.
With the risk being adopting the bias of news media.
Nobody will make anything value in public, that they didn't want to be released for free anyway.
ChatGPT's vacuum has brought back a desire for privacy and will probably contribute to destroying piracy too.
ChatGPT has destroyed the 'study hard and get reward loop' for collaborative effort on the internet. If you use chatGPT, it absorbs all your question data and gives you nothing in return. You can't commit to random people, as they are expected to leak your IP onto gpt.
Isaac Newton using chatGPT would upload the core of calculus to GPT in research questions, and see no personal benefit for doing so.
There is no greater thief in history of academic work, than electronics.