Your instinct is correct. The major legal research providers (Thomson Reuters and LexisNexis) both provide “citators”, which are human annotations of which cases and statutes have been overruled, upheld, criticized, etc. One of the issues the paper describes is the fairly ham-handed way this gets integrated into these systems, causing even more trouble.
Yes. “Superalignment” (admittedly a corny term) refers to the specific case of aligning AI systems that are more intelligent than human beings. Alignment is an umbrella term which can also refer to basic work like fine-tuning an LLM to follow instructions.
I think this misses the mark. The ad is inherently symbolic—it’s not this particular piano, but the fact that they’re destroying all of these beloved instruments of creativity in such a gratuitous and evocative manner. That’s what upsetting, not the literal fact that one piano was destroyed in the making of the ad.
Yes, compute-optimal for training only. The purpose of the paper wasn’t to determine the most economically practical model one could build, the most “intelligent” model one could build given some amount of compute.
I’ve been using the Claude 3 API since the models were announced. I believe it’s generally available (though capacity constrained & rate limited at present).
A caveat: my impression of Phi-2, based on my own use and others’ experiences online, is that these benchmarks do not remotely resemble reality. The model is a paper tiger that is unable to perform almost any real-world task because it’s been fed so heavily with almost exclusively synthetic data targeted towards improving benchmark performance.
I think if you’re looking for a logical answer from first principles, you won’t find one. It’s more that the legal system runs on precedent, and a book fits far more squarely in the fact patterns of previous First Amendment cases. Likely the source code case would end up with the same outcome, but it doesn’t hurt to make it more obvious.
That’s fair, but TypeScript coupled with modern JS dev tooling makes it actually a pretty palatable experience these days. Speaking as someone who used to want to jump into pits for JavaScript-related reasons.
The paper refers to ChatGPT as a 175B parameter LLM. This is almost certainly incorrect; the original largest version of GPT-3 was 175B but analysis of the speed and cost of the current model as well as public statements by OpenAI indicate it’s as much as 5-10x smaller.
I recently purchased a 3090 on Reddit’s hardwareswap community for $550. New GPUs are pricey right now because of shortages, but if you look around a bit it can be affordable.
I also thought that seems implausibly large. My guess, based on some quick research, is that a large fraction of that is procedurally generated, not human-written. Here's one example:
> Formal verification tools allow those teams to be confident that their solution is complete and error-free, and the code itself is then automatically generated... With such a model-driven engineering (MDE) approach, software developers at ASML recently replaced half a million lines of code that had been built the conventional way.
If you read the complaint, it explains this pretty well. The use of copyrighted content by search engines is fundamentally different from the way LLMs use that same content. The former directs traffic (and therefore $$) to the publisher, the latter keeps the traffic for itself.
The legal misconception I want to flag in your logic is the notion that all uses of the Common Crawl are equally infringing/non-infringing. If you use the Common Crawl to create a list of how often every word in English appears on the internet, that’s unquestionably transformative use. But if you use it to host a mirror of the NYT website with free articles, that’s definitely infringement. The legality of scraping is one matter, and the legality of what you do with the scraped content is quite another.
As the paper notes, two of the authors are from DeepMind, the chemicals they are synthesizing are those predicted to be stable by a DeepMind model, and the entire work arises out of an extensive collaboration between DeepMind and the Berkeley lab: https://www.nature.com/articles/d41586-023-03745-5
The main thing is chat is just one application of LLMs. Other applications are much more latency sensitive. Imagine, for instance, an LLM-powered realtime grammar checker in an editor.
Computers aren’t humans. There’s no reason licenses should treat them the same way when completely different considerations of fairness and economic impact apply.
That’s because you’re a human. Copyright and fair use law was built for people, and letting AI models have the same privileges will lead to worse outcomes. What’s so wrong about saying that it isn’t fair use for a machine to learn from your copyrighted data?
The article seems premised on a misunderstanding that OpenAI is a research lab. For all intents and purposes, it’s a for-profit subsidiary of Microsoft, and there’s little financial incentive for it to maintain old models for others’ benefit.
I think the issue with the “AI doing X better than most people is an indictment of the people or the way we teach them” genre of takes is that it assumes the current state of AI progress will hold. Today, it writes at a college freshman level, but yesterday it was at a fourth grade level. If it surpasses most or all professional writers tomorrow, what will we say?
It is very important to note that in the state of New York, the trial court (the lowest level court) is called the Supreme Court. So while this ruling is notable, it is not the state’s highest court ruling on the matter. This ruling will be appealed by the government and likely reversed on appeal.
I think the point is that Facebook's contribution to society (with its actual product, not its tech work) has been negative; its goal is to turn every knob it can find to get people to spend more time on Facebook, not actually solve any problems. All businesses are selfish, but some manage to meaningfully improve some aspect of people's lives along the way.