It's not "open enough" to do an honest evaluation of these systems by constructing adversarial benchmarks.
2,008 karma · joined August 9, 2018
> Audiences, however, must do their own figuring out. “I don’t ever explain it. Because it’s not a word thing. It would reduce it, make it smaller.” These days he rarely gives interviews, not even during the hugely hyped return of Twin Peaks last year – a show that is still debated as either the best or worst TV of 2017. “When you finish anything, people want you to then talk about it. And I think it’s almost like a crime,” he explains. “A film or a painting – each thing is its own sort of language and it’s not right to try to say the same thing in words. The words are not there. The language of film, cinema, is the language it was put into, and the English language – it’s not going to translate. It’s going to lose.”
https://www.theguardian.com/film/2018/jun/23/david-lynch-got...
GPT is confusing the fundamental theorem of calculus with the meaning of integration.
ETA: what makes LLMs so worrying is that I know calculus quite well, and I think GPT-4 is dumber than a mouse, yet I still felt gaslit by GPT-3.5's wrong answer! I had to make a mental picture to make sure I wasn't contradicting the Mighty Talking Robot. This technology is poison.
The ease with which LLMs launder misinformation at scale is genuinely dismaying.
Really not sure what your point is, it has nothing to do with Hinton's claim, and nothing to do with my comment.
It's completely irresponsible and a ton of people pointed it out! You ignoring the critics doesn't mean they don't exist.
OpenAI just straight up lied when they said Sora was able to accurately emulate physics. Sora is literally dumber than a bee! Unlike bees, it doesn't even understand object permanence! Things fade in and out of existence, change sizes arbitrarily, etc. The fact that it occasionally makes plausible-looking solid deformations and fluid motions is because ANNs are natural interpolators, and much of physics can be blindly interpolated, at least in short intervals. The idea that any of this constitutes a "simulation of physics" is childish.
- LLMs don't know personal details about your friends and family members
- LLMs don't have your personal memories from age 2 to age 63
- LLMs don't know what things feel like
- LLMs don't know what things smell like
- LLMs don't know what it's like to experience sexuality
The list goes on and on. It's particularly Hinton ignoring individual knowledge about individual social circumstances that drives me crazy. Human brains are not actually designed to store textual trivia! Why on earth would any rational AI researcher claim "knowledge about the war of 1812" means the LLM is very impressive compared to a human, but "knowledge about how my cat likes to be pet" is irrelevant?
A ton of AI researchers do not seem to care about the problems human brains actually need to solve.
Your attitude is naive and competely at odds with (honest) AI safety research. The rational thing for users is to assume DALL-E 2 is contaminated by CSAM unless OpenAI can publicly demonstrate otherwise. Their leadership is too stupid and dishonest for people to take their word.
https://ieeexplore.ieee.org/abstract/document/9423393
https://arxiv.org/pdf/2110.01963.pdf?trk=public_post_comment...
https://www.theguardian.com/technology/2023/aug/02/ai-chatbo...
If Big Tech can't even catch pornography or explicit uses of the n-word, there's no way they'll be able to filter subtle LLM / art generator hallucinations.
Keel in mind that if you had a variation of Go where there was a "hole" in the middle of the board, both Lee Sedol and a competent amateur would be able to play competent "Doughnut Go" without any prior experience. But AlphaGo and its successors would certainly make a ton of dumb unforced errors unless it practiced at least a few hundred games. (I am basing this observation on similar experiments with a similar Breakout AI, not sure if these experiments have been done with Go.)
Mammals, including humans, have advanced brains because we evolved to solve weird and unexpected problems with moderate reliability, not to optimize well-known benchmarks with high reliability. (This is also why plants are green instead of black.) By contrast, AlphaGo is a machine designed to solve a highly specific problem. The whole point of machines is that they dominate humans at specific tasks, otherwise we would just use a human. But we don't describe bulldozers as "superhuman" unless we're being intentionally obscure; the same should apply to AI. Otherwise we risk assuming the AI is capable of things it probably can't do without retraining.
I think part of the perception that a total eclipse "perfectly" blocks the sun is that the corona has a fuzzy edge but the surface does not, so when the moon is actually larger than the sun in the sky you still see the same eerie coronal glow. In particular there's little perceptible difference between blocking 100% of the sun and blocking 105% of the sun, compared to 100% versus 95%.
What I find frustrating is that Cody is doing deterministic language-specific tricks that don't depend on the underlying LLM, so why not just give a list of all the languages you did deterministic tricks for and call that the supported languages? Why be vague? Why deflect to the underlying LLM to encourage people to try your product with languages you won't do any work to support?
Claiming it works on "all programming languages" is lazy and dishonest, and clearly false. There's nothing magical about LLMs that relieves software developers of the duty to specify the limitations of their software.
But this test didn't seem to include TypeScript so it's obviously not comprehensive. I'm not convinced this information is actually in one place.
That link to the list of supported languages is broken. I couldn't find a similar file elsewhere in the repo: maybe the list got folded up into a function in another file? Also a bit annoying that I couldn't find the info on the company's website (though I gave up pretty quickly).
Say I released a linear algebra library and extolled its performance benefits + ease of development, then offhandedly mentioned that "it has some concerns with accuracy" without giving further details. You wouldn't say "ah, he focused on performance rather than accuracy." You'd say "is this a scam?" It is never acceptable for healthcare software researchers and practitioners to ignore accuracy. LLMs don't change that.
The only thing that sort of cynical laziness is good for is justifying bad decision-making by doctors and hospital administrators.
> Barriers to adoption include draft message voice and/or tone, content relevance (8 positive, 1 neutral, and 9 negative), and accuracy (4 positive and 5 negative).
That's it! Outside of that it looks like almost all the metrics are about how this makes life easier for doctors. Maybe there is more in the supplement, I have not read closely.
Look: I am not a policymaker or a physician. It's quite plausible that a few LLM hallucinations is an acceptable price to mitigate the serious threat of physician overwork and sleep deprivation. And GPT-4 is pretty good these days, so maybe the accuracy issues were minor and had little impact on patient care.
What's not acceptable is motivated reasoning about productivity leading people to treat the accuracy and reliability of software as an afterthought.
But if you're interested in the structure of plane geometry then there is no such structure-preserving isomorphism because any map into a finite set will "delete" information about side length and angles.
The structuralist idea is that interesting mathematics can be "most easily" found by considering structure-preserving isomorphisms and not the structures themselves. In particular dealing with the structures directly can obscure the mathematics you are trying to discover, e.g. dealing with a full Euclidean group when the dihedral group is all the problem requires.
Randomized Algorithms 101 - or basic stochastic reasoning - suggests that if the temperature parameter is > 0, querying an LLM N times and picking the majority result (perhaps with an N+1th query to the LLM) will generally result in better performance than asking it once and choosing that result.
It seems plausible to me that the gains can be further improved with a specialized mixture of different LLMs (which could then be run at temp = 0), or by finding better ways to break tasks into subtasks as this paper suggests. But AFAICT nobody has done anything to actually quantify these hypothetical gains versus the dumb randomized algorithm approach! In particular there might be voting strategies or mixtures - even specific models - where MoE/etc is strictly worse than naive repetition.
I am a concerned citizen w.r.t LLMs rather than a researcher, so I might be missing something. It just seems odd that LLM researchers forgot the first chapter of Motwani/Raghavan.
1) Based on how pre-verbal children learn, one nitpick is that I strongly suspect we need to give AI touch and a sense of space in order to truly understand quantity, causality, object permanence, etc.
2) Something that is not a nitpick: even a superhuman multimodal AI wouldn't have direct access to human emotions, sexuality, ideas of natural beauty, etc. I don't think humans have run out of interesting things to say about these ideas.
(In particular, I don't think a superhuman AI is capable of understanding music unless it is directly emulating the biological processes by which humans understand music. The issue is not "logical" - melodies don't actually make sense analytically.)
In general AI researchers have done a very bad job exploring how a system might be "near-human" according to some fancy linguistic benchmark, yet dramatically dumber than a pigeon in terms of general reasoning abilities.
This is especially true with these data brokers, since it's not like "cost of materials and labor + a profit margin" makes sense. In this particular case data is more like a new commodity than a manufactured good.
First of all, I would say that it's not an "artificial constraint" so much as what you're saying is an artificial mitigation. When it comes to human programmers, we expect them to be able to execute small O(n^2) and O(2^n) algorithms on paper, and if they're not able to we rightfully question whether or not they actually understand the algorithm versus just having a bunch of code memorized.
But the bigger problem is that the O(n) limitation doesn't just apply to formal math/CS problems; many natural language queries have a computational complexity, and I believe this result puts a limit on those that GPT4 can (easily) solve. I'll give a concrete example from my testing GPT-4 last year[1]. Naively GPT-4 struggled to solve two simple problems:
1) Given a pair of sentences, choose the one that has the most words.
2) Given a list of n numbers, choose the largest.
Without chain-of-thought, GPT-4 was not able to solve these problems reliably because it was clearly using spurious statistical cues and "guessing," e.g. seeming to think "antidisestablishmentarian" was always in a sentence with many words. Using chain-of-thought prompting ("let's think step-by-step") GPT-4 was able to solve these problems easily.
However, these are both O(n) problems. If you asked GPT-4 "given a list of n sentences, choose the one with the most words. Let's think step by step:" it could not reliably solve the problem! It either counted the sentences step-by-step but then guessed as to what the largest number was; or it guessed the count of the sentences and them compared the count step-by-step. Its ability to solve two separate O(n) problems did not extend to an O(mn) problem - you would need to tailor the chain-of-thought prompt and basically hold GPT's hand to get the right answer. I think this theoretical result puts my experiment on firmer ground: transformer LLMs can be trained on a variety of linear complexity problems but they cannot natively extend those results to quadratic complexity.
And yes, GPT could put this in code and get the right answer. But you'd have to train it to write a Python function instead of using its own abilities. I don't think "if the problem seems too complex, try writing a Python function" will work as a prompting strategy: it's GPT-4's inability to recognize quadratic complexity that's the root cause of the problem. (It's also why chain-of-thought isn't a cure-all - humans need to decide whether CoT is appropriate! If you use CoT prompting on a simple factual question you'll get a bunch of useless "steps.")
Going back to the scaling - the promise of scaling is that eventually big data pays off and the LLM gains some sort of generalizable understanding of the problem. But if the performance varies smoothly and predictably, then that's probably not happening - LLMs never "grok" subjects, they just screw things up less and less. E.g. if GPT-3 got a 50% on 3-digit arithmetic because its training data include 30% of all 3-digit arithmetic problems, and GPT-4 got an 80% because its training data includes 50% of all such problems, then that's a vacuous and uninteresting form of scaling - there are only ~2,000,000 such problems so this isn't as ridiculous as it might sound. Likewise if GPT-4 had the same training data but doubled its performance because it doubled the resources spent on arithmetic training. The point isn't to deny that scaling exists, it's to question whether its worth all the money and excitement. I really don't think it is - it's not even interesting.
[1] GPT-4 can solve these examples correctly now, but that's mostly a testament to the impossibility of doing reproducible investigation on closed LLMs. I strongly suspect OpenAI later trained GPT-4 on this and other "dumb" problems: certainly you could write a Python script to generate appropriate training data. The problem is you would need to generate an ~O(mn) amount of data for every pair of O(m) and O(n) problems that someone might want to ask in conjunction. I don't actually use LLMs for work, so maybe you could see how performance varies between Claude, Mixtrial, etc.
Specifically:
> To study how the parallel structure of transformers might limit their capabilities, the pair considered the case where transformers didn’t feed their output back into their input — instead, their first output would have to be the final answer. They proved that the transformers in this theoretical framework couldn’t solve any computational problems that lie outside a specific complexity class. And many math problems, including relatively simple ones like solving linear equations, are thought to lie outside this class.
> But no matter how a prompt is phrased, as long as it causes a language model to output step-by-step solutions, the model can in principle reuse the results of intermediate steps on subsequent passes through the transformer. That could provide a way to evade the limits of parallel computation... They quantified how that extra computational power depends on the number of intermediate steps a transformer is allowed to use before it must spit out a final answer. In general, researchers expect the appropriate number of intermediate steps for solving any problem to depend on the size of the input to the problem. ...Indeed, Merrill and Sabharwal proved that chain of thought only really begins to help when the number of intermediate steps grows in proportion to the size of the input, and many problems require the number of intermediate steps to grow much larger still.
So there’s a large space of O(n) and O(constant) problems that LLMs seem to scale very well on, but if they need O(n^2) examples in their training data to learn how to solve O(n^2) problems (even with prompt engineering!) then that’s a hard limit.
ETA: there’s also good (IMO irrefutable) evidence that AI researchers have deluded themselves about the scalability of LLMs because of motivated statistical reasoning: https://arxiv.org/abs/2304.15004
In fact it seems to me that “LLMs scale well” means “LLMs can solve O(n) problems if you give them O(n) examples in their training data,” which is actually not very interesting. The interesting thing is that the internet + Moore’s law allows us to build (and compute on) O(n) training data for an enormous variety of problems.
There was always a horrible gap in this law - though less with educated workers like you suggest, and more blue-collar workers who get promoted to low-paid management. Now that white-collar workers have a more competitive job market, it is affecting us as well.
But last year I repeatedly saw ChatGPT-4 respond token-by-token much more slowly than a human would! E.g. several seconds between words. It was clearly not a human responding: at least a few times I was testing on preschool counting questions and GPT-4 was not able to answer them. I interpreted the slowness as GPT's poor quantitative reasoning. But what you're saying is simply not true, sometimes ChatGPT-4 is (or was) extremely slow.
Regardless, if OpenAI was running this con it probably wouldn't have been real-time humans writing. First of all it might be enough to have a human in the "mixture of experts" who decides the best of multiple responses when GPT-4 is unable to come to an automated conclusion. But humans could be writing ChatGPT responses due to a quirk in their UX:
- ChatGPT errors out on a certain question and asks you to try again later, as it does (or used to do) frequently
- the response is prepared by the human contractor while the user waits patiently for ChatGPT to resolve its technical difficulty
- when the user asks again ChatGPT can largely read off the answer, using its (genuine) language-processing abilities to handle variations in phrasing/etc
Also anecdotally: despite the embarrassing scam of "AI detectors," many kids did not actually get away with cheating via ChatGPT last year. Issues like fictional citations, "as a large language model I cannot," and making up facts affect high school English papers just like legal filings or scientific articles about rats. And unlike scientific peer review, high school teachers usually read things closely and notice when something isn't right. You don't need an AI detector to be suspicious when an impeccably well-written essay discusses events in Great Expectations that didn't happen.
Regardless of how shady OpenAI's aims may have been, I genuinely think the root cause of this is that someone at OpenAI foolishly used ChatGPT to automate boring tax paperwork. ChatGPT decided "John Q Vesper" or whatever was a statistically plausible name for the CEO, and this dumb mistake wasn't caught by a human because nobody wants to read tax paperwork if they think a magical talking robot is capable of what seems like a routine task. I am assuming OpenAI didn't intend to tell a ridiculous and easily falsifiable lie in its tax filings (especially if that lie contradicted their public explanations about Altman's management of the fund!). OpenAI probably wanted the paperwork to say "Sam Altman."
FWIW the IRS is generally forgiving of good-faith tax errors, but I would suspect "we spent so long lying about our chatbot that we ourselves forgot that it's dumber than a pigeon and doesn't actually understand human language" doesn't count. Considering how many two-bit lawyers got in significant legal trouble for relying on ChatGPT hallucinations, it would be outrageous if OpenAI manages to get off scot-free here.