AlphaCode as a dog speaking mediocre English
scottaaronson.blog
scottaaronson.blog
We live in exciting times. Given exponential rates of progress, I can’t really even imagine what breakthroughs we will see in the next year, let alone the next five years.
EDIT: I forgot to mention: having GitHub/OpenAI tools like CoPilot always running in the background in PyCharm and VSCode has in a few short months changed my workflow, for the better.
As you type in comments or code, CoPilot will sometime autocomplete up to about 10 lines of code, based on the content of the file you are editing (maybe just the code close to the edit point?).
My other tools? I use LispWorks Professional, Emacs with Haskell support, and sometimes IntelliJ. I work a lot on remote servers so I depend on SSH/Mosh and tmux also.
Kind of like regular autocomplete. If it's wrong you just keep typing.
We've been on the steep part of an S-curve.
AlphaZero is already capable of optimizing itself in the limited problem space of Chess.
Infinitely increasing the computing power of this system won't give it properties it does not already have, there is no singularity point to be found ahead.
And I am not sure that there are any singularities lying ahead in any other domains with the current approach of ML/AI.
We've already seen in games with simple rules and win conditions that giving computers data on what we think are good human games can make them perform worse than not giving them data. Most problems aren't possible for humans to ebncapsulate perfectly in a set of rules and win conditions to just leave the processing power to fill in the details, and whilst curating data and calibrating learning processes is an area we've improved hugely on to get where we are with ML, it's not something where human knowledge seems more likely to reach an inflection point than hit diminishing returns.
I think this is only true because, so far, our heuristics are not that clever.
The number of orders of magnitude remaining, is not that large.
But I still have hope for a undivided world, that one day even abolish the patent system. Then I would see potential for exponentional progress.
Come on, you could have said the same thing about excel 30 years ago.
The exponential model is something close than what is behind Ray Kurzweil reasoning about the Great Singularity and how the future will be completely different and we're all going to be gods or immortals or doomed or something dramatic in that vein.
The S-Curve is more boring, it means that the future of computing technology might not be that mind blowing after all, we might already have reaped most of the low-hanging fruits.
A bit like airplanes, or space tech you know, have you seen those improving by a 10x factor recently?
Is space tech included ironically here? There's been 10x (or more) improvements across dozens of space tech problems in the last couple decades.
Would that be 1000x, 100x or even 10x easier than 60 years ago? Well, I don't think so.
From my perspective space tech has never quite leaved the prototype phase, for some reason.
Generally the people using regexes care if they're correct. Frequently, all possible input variants are not enumerated in tests. Frequently, companies want to have confidence in their production code. Imagine this regex is deployed on a sign up flow, and its failures invisibly increase your churn rate. Can it happen with a hand crafted regex? Yes, of course. But I'd imagine it will happen even more frequently with an AI produced custom regex plus a person who doesn't actually understand regexes.
If you have to be a programmer to understand why something's bad, who's going to prevent it? This is a major unsolved problem in the structure and organization of working.
(1) The product specification and use cases are so poorly defined that I need to anticipate the use cases, design a system that is general enough to accommodate them, and implement it in a way that is easily changeable to accommodate the future departures from my assumptions.
(2) As I do this, I need to consider the existing systems that I'm building on top of to ensure there is no regression when this feature rolls out
(3) I consider the other teams that are doing similar work and make judgement calls about whether to write independent systems that do one thing each, or to collaborate on a general-enough system with multiple team ownership.
(4) The tech that I use to implement this must be within the narrow slice of company-sanctioned tech.
(5) I weigh constant tradeoffs on speed to market, maintainability and ownership.
I'm sure there's more, but this stuff is _hard_. If autonomous driving for white collar work is coming, as put forth by comments here, I'd like to see indications that the actual hard part of the job is in jeopardy of being executed effectively.
Maybe I don't want to believe it, so I can't see it. I'll grant that. But I truly do not see it.
But, your post makes me realize that a job could be threatened by an AI just superficially emulating a worker, because its existence might exploit the social vulnerabilities of the workplace, in spite of having no general intelligence nor exploitative social skills of its own.
Would the demand for junior developers evaporate if more experiences people can 10x daily LoC productivity? Or the other way around? Would languages with higher level abstractions (e.g. comparable to Scala if not Haskell) win or something like JS dominate?
I'm afraid they do not understand the size of problem/solution set. Suppose problem and solution are 1000 characters long and there's a set of 32 characters. Then a model is function F: X -> X where X is a set of 2^5000 elements. There are 2^10000 such functions, and the goal of a training process is to find the best one.
Training set for a language model would be well under 1 PB. So a task here is to use a training set of size 2^50 to find a function from 2^10000 space.
It's obvious that no amount of brute-forcing can possibly find this. An no classic mathematical statistical modeling can possibly help. A problem like this can be only approached via ANNs trained through backpropagation, and only because this training process is known to generalize.
Personally I only grow skeptical when claims are made about some vague Artificial General Intelligence or some other magic which no amount of inference is gonna give us.
Obviously it's tough to find such algorithm that works for all types of data/embeddings.
Do you consider programming math? Since technically you write a function which deals with 0s and 1s.
I do think it is interesting philosophically how much powerful machine learning algorithms are over the traditional linear models. And perhaps it is a good description to say that “best fit program” is inferred. However these algorithms are still limited to inference. And—while useful—calling it intelligent is not very accurate—at least in the way most people use the word—no matter how complected of a construct they are inferring.
It is in this sense that it is a brute-force approach because we should like to know the underlying model, but instead we can throw a hugely over-parameterised but relatively simple model at a problem and it will learn (statistically) the underlying phenomenon. Like you say it does much better than a combinatorial brute-force approach.
1. That we are unimpressed.
I'm gobsmacked.
2. That we don't think these are significant advances.
They're obviously huge advances.
3. That we don't think these models will have practical applications.
It's hard to imagine they won't.
4. That we think these systems are rubbish because they get things wrong.
I'm a programmer. I make mistakes all the time.
Having countered those views, the article then seems to imply that it follows that "we’ve now entered a world where 'programming' will look different." As someone who makes a living writing software I obviously have an interest in knowing whether that's true. I don't see much evidence of it yet.
These systems are certainly not (yet) capable of replacing a human programmer altogether, and whether they could ever do so is unknown. I'm interested in the implications of the technologies that have been developed so far - i.e. with the claim that "we've now entered a world..." So the question is about how useful these systems can be for human programmers, as tools.
The reason I'm skeptical of it is that the only model I've seen so far for such tooling is "have the machine generate code and have the human review it, select from candidate solutions and fix bugs". The problem is that doing so is, I expect, harder for the human than writing the code in the first place. I've mentioned this concern several times and not seen anybody even attempt to explain to me why I'm wrong about it. For example, at [1] I pointed out why some generated solutions for a particular problem would have only made my job harder and got accused of "screaming at a child for imperfect grammar."
Reviewing and fixing code is harder than writing it. Please explain why I'm wrong about that (it's certainly true for me, but maybe most people don't feel that way?), why it won't be a problem in practice or what planned applications there are for these technologies that would avoid the problem.
Please don't accuse me of cruelty to dogs or children.
You know how you can break writing down into 2 phases: creative and edit?
What if you did the creative part, braindumping all your ideas and vision into a word document. Then had an AI edit your incoherent story into engaging prose.
I find the creative part easy, but the editing part tedious.
Like, a doctor I know asked me to write a review for him. I did the braindump part, but have been procrastinating on the edit part.
I get with code it’s a bit less so, but still, there are plenty of design patterns which basically cover a lot of the output space…
Not saying it can't help you with getting an idea what to describe, but this is the kind of "editing" that it won't be good at.
You get the tool to brain dump for you, then you edit after
I get that you have to do subsequent editing, because it’s not perfect.
> Post: I realize that it “only” solves about a third of the contest problems, making it similar to a mediocre human programmer.
If this is what academia perceives software development to be then it's no wonder we have software that is so disconnected from the human problems it aims to solve.
Programmers don't routinely [re-]invent complex algorithms. We parse complex and contradictory requirements from humans, and compile them into (hopefully) simple solutions on a computer.
The solutions to "competition programming" problems are a Google search away. If you want to take it up as a professional sport then, sure, AI might just replace you (as it already has done with many other mind sports such as chess).
At the very least, I don't think it being true for you (or me) means much. Presumably like me, you learned primarily to write code and have a lot of experience with it, and have gotten better and better at that with time. I assume that if I had spent as much time and effort learning to efficiently review and fix code as I have spent learning to write it that it would be much easier.
I certainly know of artists who have shipped video games they authored alone, who wouldn't call themselves programmers or know where to start programming something from scratch, who got by with a high level engine and copy and pasting starting points from the internet. (I would call them programmers, but their style is much closer to review and edit than my own.)
Programming is mostly about writing boilerplate code using well-known architectural patterns and technologies, nothing extraordinary but which takes time (at least in my experience). If I can describe a project in a few abstract words, and the AI generates the rest, it can considerably improve my productivity, and I don't think it's going to be harder to review than code written by a junior dev who also makes mistakes and doesn't usually get it right on the first try anyway (given the AI is pretrained to know what kind of architecture we prefer). I can envision a future where programmers are basically AI operators who iterate on the requirements with the client/stakeholders and let AI do the rest. It looks like we're almost there (with GitHub Copilot and all), and I think it's enough to "revolutionize" the industry, because it changes the way we approach problems and makes us far more productive with less effort.
I have worked in large decade-old codebases and sometimes the code is truly puzzling. Usually, an old-timer helps out when you run into a block and explains design decisions taken by people who moved on. This is the crucial factor that determines how long a task takes. A task could be resolved in a few hours with the help of a senior engineer, which otherwise could take days to weeks.
All said, I think simply generating the code will not be enough to replace programmers. Now, if an AI can generate code AND explain queries about logic, syntax, etc, the game is over.
People can certainly take this approach and build tools. That might take different shapes and forms. There are many ways these models can be specialized, fine-tuned, combined with other approaches, etc.
For example, somebody can try to apply them to a narrow use case, e.g. generate front-end code in React/JS and CSS from a backend API. They can fine-tune it on best example of React code, add a way to signal uncertainty so that specification can be clarified, etc.
Nobody expects these models to be able to write an entire OS kernel any time soon. But a lot of types of programming are far more regular, repetitive and can be verified a lot easier.
The best characterization of such massive language models I've seen is "world class bullshitters": they are very good at producing output that superficially looks plausible, but neither know nor care whether what they're saying has any relationship to the truth.
This is virtually guaranteed to make such code reviews very frustrating, and of course AlphaCode & co have no capability of explaining why they wrote a certain line. I can't see this having much of a role in a high quality code base, but I suspect the sort of management which currently would offshore coding to the lowest bidder would be quite enamored of it.
For example, self-driving cars are also currently in this weird place where they're prone to making lots of errors, so a simple task for the human (driving) is replaced with a much more taxing one (keeping an eye on an AI driver and being ready to react at any moment), if we want to do it safely.
Whether any of those applications will be able to progress past that weird space into "no/minimal supervision needed" remains an open question.
We're also in the very very early days of code generation models. Even I can see some ways to improve AlphaCode:
- the generate->cluster->test process feels like a form of manual feature engineering. This meta layer should be learned as well, possibly with RL
- programming is inherently compositional. Ideally it should perform the generate->cluster->test step for each function and hierarchically build up the whole program, instead of in a single step as it does now
- source code is really meant for humans to read. The canonical form of software is more like the object code produced by the compiler. You can probably just produce this directly
We programmers and data scientists might find ourselves among the first half of knowledge workers to be replaced and not among the last as we previously thought.
You can write code that is valid as in "can be compiled" but outside of C++ standard. It is duty of programmer to not have those, as compiler usually assumes that there's no UB in your code and can do unintuitive things with optimizations.
e.g
int foo(int8_t x) {
x += 120
return x;
}
int bar(int8_t y) {
int z = foo(y);
if (y > 8) {
do_important_thing(z);
}
}
1. signed overflow is a UB. Compiler than assumes that everything passed to foo is less than 8;
2. We pass y to foo => y < 8
3. if branch can then be optimized out
I was too lazy to check the exact value of 2^31 and payed for it.
Instead of semantically correct Python, programmer and data scientists’ jobs will be to work in semantically correct English. Fundamentally the job won’t change (you’ll be programming the AI rather than program the machine directly).
And specifying a full app to be autocoded is most certainly NOT a solved problem.
Until AutoCode can build an app that employs compound AND complex behaviors, like Angry Birds, or a browser, I'll continue to see it as little more than a write-only copy/paste/derive-driven macro generator.
That doesn’t mean you shouldn’t ever use ORMs, or that in simple cases they aren’t “good enough”. But at some level of complexity it breaks down.
AI-assisted programming is the new leaky abstraction.
They key advantage of producing source code is that you can usually tell what the produced program does.
Because the whole thing is, a dog’s abstract mental capabilities are far below a human — thus why it would be ASTOUNDING that a dog could master even a primitive form of speaking English.
On the other hand, here we are brute forcing a solution from analyzing millions of man-years of published English speech, by using a huge array of computing power to precompute various answers and sift them.
It is a bit like “solving checkers” and then claiming “wow, a dog could have done this”. It is like making a cheat sheet for a test by analyzing all answers ever given, and then summarizing them in a vector of 102827 dimensions, and claiming that it is the same as coming up with clever and relevant one liners on the spot using the brain of a dog.
NO. It’s not nearly as astounding or impressive.
The vocal apparatus is not there, but there is certainly more cognition than what people think dogs have (there's a question that I wonder if language enables thought or if thought enables language)
And this is even something that a cat can do (not as much language skills, but there's thought going on) https://www.youtube.com/c/BilliSpeaks
This NYT article at least does not discredit it [0]. Have you looked more into it? Do you think it would be useful to train your dog to do it?
[0]https://www.nytimes.com/2021/05/27/style/bunny-the-dog-anima...
Even being able to distinguish those would be a "win".
The Bunny video that I still find the most useful communication is https://youtu.be/6MMGmRVal6M
edit: or just have a conversation of any kind longer than a 2 minute video. Or one without the owner in the room, where she talks back with the dog using the same board. That would at least be amenable to Turing.
edit2: here's a test - put headphones on the owner and block her vision of the board. Sometimes pipe in the actual buttons the dog is pressing, other times pipe in arbitrary words. See if the owner makes sense of the random words.
Consider this older one where there weren't as many words - https://youtu.be/6MMGmRVal6M and https://youtu.be/FPC6ElzSdxM
Or for billie the cat - https://youtu.be/DiuQqgTw-jY
The point with these is that there is language related thought going on there and an attempt to communicate from the dog (or cat) to the human.
1) just seemed like random pressing until the dog pressed "paw", then the owner repeated loudly "something in your paw?" The dog presented its paw, then the owner decided "hurt" "stranger" "paw" was some sort of splinter she found there. The dog wasn't even limping.
2) I didn't get any sense of the presses relating to anything the dog was doing, and since the owner was repeating loudly the thing she wanted the dog to find, I was a bit surprised. Then the dog presses "sound," the owner connects this with a sound I can't hear, then they go outside to look for something I can't see.
Billie the Cat: I simply saw no connection between the button presses and anything the cat did. The cat pressed "outside" but didn't want to go outside. The cat presses "ouch noise" and the owner asks if a sound I didn't hear hurt her ears. Then the cat presses "pets" and the owner asks if the cat wants a pet? The cat presses "noise" and the owner continues the monologue apologizing for the painful noise and offering to buy her cat a pet. Sorry to recount most of the thing, but I don't get it at all.
-----
Not trying to debunk talking pets, but I'm not seeing anything here. I at least expected the dog to be smart enough to press particular buttons for particular things, but I suspect the buttons are too close together for it to reliably distinguish them from each other. I'd be pretty easy to convince that you could teach a dog to press a button to go outside, a different button when they wanted a treat, and a different button when they wanted their belly rubbed. In fact I'd be tough to convince that you couldn't teach a dog to do that. Whatever's being claimed here, however, I'm not seeing.
-----
edit: to add a little more, I'm not even sure that *I* could reliably do what they're claiming the dog is doing. To remember which button is which without being able to read is like touch typing, but worse because the buttons seem to be mounted on re-arrangeable puzzle pieces. Maybe I could associate the color of those pieces with words, but that would only cover the center button on each piece.
If a dog were using specific buttons for language (or if I were doing the same thing) I'd expect the dog to press a lot of buttons, until he heard the sound he was looking for, then to press that button over and over. Not to just walk straight to a button and press.
I think the cat just presses the buttons when it wants the owner to come, and presses them again when the owner says something high pitched at the end and looks at it in expectation.
In my opinion, more ridiculous claims have already been proven by science (for example Quantum Mechanics).
Also you have to make a distinction between the optimizing process (evolution/training neural nets) and the intelligent agent itself (human/machine intelligence).
The problem I have is comparing that to having a dog speak English. It’s totally wrong. You had access to all these computing resources, and the sum total of millions of work by humans. You didn’t bootstrap from nothing like AlphaZero did, but just remixed all possible interesting combinations, then selected the ones you liked. And you try to compare this “top down” approach to a bottom-up one?
The top down approach may give BETTER answers and be MORE intelligent. But the way it arrives at this is far less impressive. In fact, it would be rather expected.
Current machine learning models have around ~100B parameter, human brain has ~100T synapses. Assuming one DNN parameter is equivalent to 1 synapse, then the biggest models are still 1000 times smaller than human brain.
Cat or dog would have around ~10T synapses.
AlphaCode has ~50B parameters, that is 20 times less than number of synapses in a mouse brain ~1T. Honey bee has ~1B synapses.
So AlphaCode would be somewhere between a honey bee and a domestic mouse.
https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
For example, given 3 beads and 2 stacks for the 10er and 1er positions of a number. One can make 4 different numbers when stack all the beads like in an abacus. Without using all the beads, one can of course make more numbers. The question is how many different numbers one can make using n beads, giving full and partial usage of the beads.
It's indeed a very simple problem for 5-7 year olds and I hardly think anyone cannot solve it. Nevertheless, I seriously doubt AlphaCode can solve the problem. What does it say about its supposed Intelligence?
Any leetcode grinder can tell you there are a few dozen types of competitive programming problem (monostack, breadth-first state search, binary search over solution space, etc.) so solutions to new problems are often very similar to solutions for old problems.
The training corpus for these code transformers is so large that almost all evaluation involves asking them to generate code from the training corpus.
To evaluate CoPilot, we should ask questions that are unusual enough they can't be answered through regurgitation of the training corpus.
What does CoPilot generate, given this prompt:
// A Go function to set the middle six bits of an unsigned 64-bit integer to 1.
func six1s(x uint64) uint64 {
const (
ones = uint64(63)
shift = (64 - 6) / 2
mask = ones << shift
)
return x | mask
}
Here's what it generated for me:
// A Go function to set the middle six bits of an unsigned 64-bit integer to 1.
func set6(x uint64) uint64 {
return x | 0x3f
}CoPilot is helpless if it needs to do more than just regurgitate someone else's code.
The training of these models on GitHub, so they regurgitate licensed code without attribution, is the greatest theft of intellectual property in the history of Man. Perhaps not according to the letter of the law, but surely according to the spirit.
CoPilot regurgitated somebody's solution... to a different problem. It's pathetic.
func set6(x uint64) uint64 {
return x | 0x7E0000000
}
CoPilot got it WRONG. You got it RIGHT.
You UNDERSTAND the problem but CoPilot does NOT.
Is that clear?
Prompt:
// Function to set bits 29-34 in a uint64 to 1
func setbits (uint64 x) uint64 {
return x | (1 << 29) | (1 << 30) | (1 << 31) | (1 << 32) | (1 << 33) | (1 << 34)
}
You spelled out exactly what to do, in term of what it has seen in its training, and it was able to regurgitate a solution.
By asking question that require mathematical reasoning or are too far from the training corpus, I can create an endless list of simple problems that CoPilot can't solve.
Look at my comment history to see another one (swapping bits).
This makes it easier to match the code against the specification, something a random (and in this case even wrong) magic number fails to do.
But maybe I'm overthinking it.
You're correct.
Exactly! It's great that Copilot can generate correct code for a given question, but we cannot gauge its full capability unless we try it on a range of different questions, especially ones that are not found in the training data.
I mentioned this in the other AlphaCode post: It would be nice to know how "unusual" a given question is. Maybe an exact replica exists in the training data, or a solution exists but in a different programming language, or a solution can be constructed by combining two samples from the training data.
Quantifying the "unusual-ness" of a question will make it easier to gauge the capability of models like AlphaCode. I wrote a simple metric that uses nearest neighbors (https://arxiv.org/abs/2109.12075). There are also other tools to do this: conformal predictors, which are used in classification methods, and the RETRO transformer (https://arxiv.org/pdf/2112.04426.pdf) has a calculation for the effect of "dataset leakage".
It does not seem to be copying from the training data in any meaningful way.
My point is, I would like to verify this claim with different metrics, because we probably have different interpretations of the word "meaningful".
AlphaCode measures similarity between programs via longest common substrings. That's better than nothing, but that would mean two programs that differ only in variable naming would not be considered similar. If two programs differed only in the names of the variables/functions, I would consider that copying.
I think there are better comparisons of structural similarity: compare the ASTs, or the bytecode/assembly code generated, the control flow graphs, or perform SSA and compare the blocks generated. Each of these might have weaknesses as well, but they won't be as obvious as variable renaming, and so we'd get a better idea of what AlphaCode is copying, and therefore a better idea of its full capabilities.
I expect AlphaCode performs well on Python because the training data is dominated by Python, but Python isn't ideal for comparing program structure. I wonder which programming language (given enough training data) would be best suited for language model generation and analysis.
Why do you think Python isn't good for comparing program structure? Certainly you could compare ASTs and bytecode pretty easy; I think it's actually much easier to do so than in C/C++ since you don't have to deal with preprocessor junk and compile flags influencing the meaning of the code. There's less available for classical analyses like data and control flow analysis, in part because those are much harder in dynamic languages, but there are some tools out there like the ones used in PyPy for getting SSA CFGs [1].
I feel like many people are equivocating a bit on what they mean by "regurgitating" here. It seems clear that the models have at best a shaky grasp on code semantics (e.g., they are bad at things like predicting what the output of some code will be: https://arxiv.org/abs/2112.00114), and struggle with problems that are very different from anything they've seen before.
[1] https://foss.heptapod.net/pypy/pypy/-/tree/branch/default/rp...
Agreed.
> Why do you think Python isn't good for comparing program structure?
From recent experience I prefer control flow analysis, or something that results in a graph structure. As you said, that's harder with dynamic languages. I also think some Python features (english-like syntax, f-strings, division converts int to float, whitespace indentation, loops vs generator expressions) make structural comparisons messy, but that may just be bias.
The ideal language would be one with minimal syntax, where we can target a decent range of programs, and obtain as much info as possible about program structure without actually running the program. I've come across LISP-without-macros in Dreamcoder (https://arxiv.org/abs/2006.08381), BF++ (https://arxiv.org/abs/2101.09571), and a couple of others which I can't remember right now. I think the APL family (APL/J/K) would interesting because fewer characters to generate, but each character has a lot of meaning.
Right now I'm looking at flow-based programming (FBP) for this: In FBP the control flow is explicit - the program code describes a directed acyclic graph (DAG), so comparing program structure becomes straightforward (subgraph isomorphism with some heuristics). I'm writing a toy FBP language that draws images (https://github.com/mayahq/flatland), with which I aim to test what these models understand.
That's 25x in one go. Maybe we have the chance to run GPT-4 ourselves and not need 20 GPU cards and a 1mil $ computer.
[1] https://deepmind.com/research/publications/2021/improving-la...
Measuring the Algorithmic Efficiency of Neural Networks: https://arxiv.org/abs/2005.04305
In this work, we argue that algorithmic progress has an aspect that is both straightforward to measure and interesting: reductions over time in the compute needed to reach past capabilities. We show that the number of floating-point operations required to train a classifier to AlexNet-level performance on ImageNet has decreased by a factor of 44x between 2012 and 2019. This corresponds to algorithmic efficiency doubling every 16 months over a period of 7 years. By contrast, Moore's Law would only have yielded an 11x cost improvement. We observe that hardware and algorithmic efficiency gains multiply and can be on a similar scale over meaningful horizons, which suggests that a good model of AI progress should integrate measures from both.
Mastering Atari Games with Limited Data: https://arxiv.org/abs/2111.00210
Reinforcement learning has achieved great success in many applications. However, sample efficiency remains a key challenge, with prominent methods requiring millions (or even billions) of environment steps to train. (...) This is the first time an algorithm achieves super-human performance on Atari games with such little data. EfficientZero's performance is also close to DQN's performance at 200 million frames while we consume 500 times less data. EfficientZero's low sample complexity and high performance can bring RL closer to real-world applicability.
500x improvement over ~10 years since DQN that roughly 2x improvement in sample complexity every year.
A Time Leap Challenge for SAT Solving: https://arxiv.org/abs/2008.02215
We compare the impact of hardware advancement and algorithm advancement for SAT solving over the last two decades. In particular, we compare 20-year-old SAT-solvers on new computer hardware with modern SAT-solvers on 20-year-old hardware. Our findings show that the progress on the algorithmic side has at least as much impact as the progress on the hardware side.
AI research has also tiny budgets compared to the biggest scientific projects:
GPT-3 - $0.01B
LIGO - $1B
LHC - $7.5B
JWST - $10B
ITER - $20B
ISS - $150B
So even conservative estimates put the compute cost alone at least one order of magnitude higher than your figure of $10M.
[0] https://lambdalabs.com/blog/demystifying-gpt-3/#:~:text=But%....
[1] https://venturebeat.com/2020/06/01/ai-machine-learning-opena...
The machine learning techniques that were developed and enhanced during the last decade are not magical, like any other machines/software.
But people have different and irrational expectations about AI.
The AI that has taken over google search has made a great product kinda of awful now.
What breakthroughs are you referring to?
I mean Photoshop could do that 10+ years ago without "machine learning" even being a thing people talked about.
Photoshop definitely cannot do that, I know that for a fact.
https://towardsdatascience.com/virtual-background-for-video-...
There's an example article on the subject.
I feel the same principle should apply to software - yes, you've built a mathematical model which can take input and make a decision based on the internal algorithms, if you trained it to detect background in video then that's what it will do.
But it's not intelligence. It's no different than the bacteria deciding what to eat because certain biological receptors were triggered. I think calling it intelligent is one of the biggest lies IT professionals tell themselves and others.
That's not to say the technology isn't impressive - it certainly is. But it's not AI in my opinion.
Take autonomous cars. Sure, Musk is over-hyping, but we are making progress.
I imagine it will go something like:
Step 1) support for drivers (anti sleep or colision).. done?
Step 2) autonomous driving in one area, perfect conditions, using expensive sensors
Step n) gradual iteration removes those qualifications one by one
.. yes, it will take 10/20 years before cars can drive autonomously in chaotic conditions such as "centre of Paris in the rain". But at each of those steps value is created, and at each step people will say "yea but..".
We had real time green screens 10 years ago. I don't think it's that unimaginable.
I've implemented a natural language search using bleeding edge work, the results I can assure you are impressive.
Everything from route planning to spam filtering has seen major upgrades thanks to ML in the last 8 years. Someone mentioned the zoom backgrounds, besides that image generation and the field of image processing in general. Document classification, translation. Recommendations. Malware detection, code completion. I could go on.
No one promised me AGI so idk what you're on about and that certainly wasn't the promise billed to me when things thawed out this time but the results have pretty undeniably changed a lot of tech we use.
"Our long term aim is to solve intelligence, developing more general and capable problem-solving systems, known as artificial general intelligence (AGI)."
Dipping into my own speculation, to your point about how to measure, between our (humanity's) superiority complex and with how we move the baselines right now I don't know if people will acknowledge AGI if and until it's far superior to us. If even an average adult level intelligence is produced I see a bunch of people just treating it poorly and telling the researchers that it's not good enough.
Edit: And maybe I should amend my original statement to say I've never heard a researcher promise me about AGI. That said that statement from DeepMind doesn't really promise anything other than they're working towards it.
https://www.vetta.org/2011/12/goodbye-2011-hello-2012/
He doesn't say so publicly any more, but I think it is due to people's negative reaction. I don't think he changed his opinion about AGI.
You may have implemented something that impressed you but when you move that solution into real use were other's as impressed?
That's what is probably happening with the google search team. A lot of impressive demos, pats on the back, metrics being met but it falls apart in production.
Most people don't think Google's search is good. Most people on Google's team probably think it's better than ever. Those are two different groups.
Spam filtering may have had upgrades but it is not really better for it and in many cases worse.
I used it in real use, the answer was still a hard yes.
My point is - if you don't actually work with state of the art AI research, then yeah, it's easy to see it as nothing more than overhyped garbage, because that's exactly what's being sold to regular consumers.
[1] https://ai.plainenglish.io/how-ai-actually-helped-in-the-dev...
The fact that the researchers were clear about that doesn't absolve the marketing department, CEOs, journalists and pundits from their BS claims that we're doing something like AGI.
Backpropagation has zero creativity. It's an elaborate mechanical parrot, and nothing more. It can never relate to you on a personal level, because it never experiences the world. It has no conception of what the world is.
At least a dog gets hungry.
Not persuaded? Try https://news.ycombinator.com/item?id=23346972
The problem is: it's not really clear how much creativity we have, and how much of it is better explained by highly constrained randomized search and optimization.
> It can never relate to you on a personal level
Well, sure. Even if/once we reach AGI, it's going to be a highly alien creature.
> because it never experiences the world.
Hard to put this on a rigorous setting.
> It has no conception of what the world is.
It has imperfect models of the world it is presented. So do we!
> At least a dog gets hungry.
I don't think "gets hungry" is a very meaningful way to put this. But, yes: higher living beings act with agency in their environment (and most deep learning AIs we build don't, instead having rigorous steps of interaction not forming any memory of the interaction) and have mechanisms to seek novelty in those interactions. I don't view these as impossible barriers to leap over.
AlphaGo is grounded because it experienced Go, and has a very good conception of what Go is. I similarly expect OpenAI's formal math effort to succeed. Doing math (e.g. choosing a problem and posing a conjecture) benefits from real world experience, but proving a theorem really doesn't. Writing a proof does, but it's a separate problem.
I think software engineering requires real world experience, but competitive programming probably doesn't.
You might be using a different definition of "magical" than what others are using in this context.
Of course, when you break down ML techniques, it's all just math running on FETs. So no, it's not extra-dimensional hocus pocus, but absolutely nobody is using that particular definition.
We've seen unexpected superhuman performance from ML, and in many cases, it's been inscrutable to the observer as to how that performance was achieved.
Think move 37 in game #2 of Lee Sedol vs. AlphaGo. This move was shocking to observers, in that it appeared to be "bad", but was ultimately part of a winning strategy for AlphaGo. And this was all done in the backdrop of sudden superhuman performance in a problem domain that was "safe from ML".
When people use the term "magic" in this context, think of "Any sufficiently advanced technology is indistinguishable from magic" mixed with the awe of seeing a machine do something unexpected.
And don't forget, the human brain is just a lump of matter that consumes only 20W of energy to achieve what it does. No magic here either, just physics. Synthetically replicating (and completely surpassing) its functionality is a question of "when", not "if".
Maybe we should start by "humans are the greatest thing ever" and then try to fit our world knowledge to that conclusion. We feel it right in our qualia that we are right, and qualia is ineffable.
That's not my understanding of the 'hard problem of consciousness'. Admittedly, all I know about the subject is what I've heard from D Chalmers in half-a-dozen podcast interviews.
Can you point to a definitive source?
AlphaGo is nothing else than brute force.
And brute force can go a long way, it should not be underestimated.
But so far, this approach has not let to emergent behaviors, the ML blackbox is not giving back more than what was fed.
They used a fantastic amount of compute for their solution, but, as has uniformly been the case for neutral networks, the compute required for both training and inference has dropped rapidly after the initial research result.
This statement is completely false with accepted definitions of "brute force" in the context of computer science.
I think people are completely wrong to pooh pooh the utility of computers being better at search and calculations in an ever wider range of applied fields, but linking computers surpassing humans at more examples of those problems to certainty we'll synthetically replicate brain functionality we barely understand is the sort of stretch which is exactly why AGI-sceptics feel the need to point out that this is just a tool iterating through existing programs and sticking lines of code together until the program outputs the desired output, not evidence of reasoning in a more human-like way.
eg, for speech compression: https://arxiv.org/abs/2107.03312
It's much more simple, and more deep: most humans believe they are special/unique/non-machine like spiritual beings. Anything suggesting they could be as simple as the result of mechanical matrix multiplications is deeply disturbing and unacceptable.
There is a rich recent anti-AGI literature written by philosophy people which basically boils down to this: "a machine could never be as meaningful and creative as I am, because I am human, while the AGI is just operations on bits".
Life, as it exists on Earth in the form of men, animals and plants, is to be found, let us suppose in a high form in the solar and stellar regions.
Rather than think that so many stars and parts of the heavens are uninhabited and that this earth of ours alone is peopled—and that with beings perhaps of an inferior type—we will suppose that in every region there are inhabitants, differing in nature by rank and all owing their origin to God, who is the center and circumference of all stellar regions.
Of the inhabitants then of worlds other than our own we can know still less having no standards by which to appraise them.
Nicholas of Cusa, c1440; Cardinal and Bishop
I presume the amount of hype AI research has been getting for the past 4 decades might be at least part of the reason. I also think AI is terribly named. We are assigning “intelligence” to basically a statistical inference model before philosophers and psychologists have even figured out what “intelligence” is (at least in a non-racist way).
I know that both the quality and (especially) the quantity of inference done with machine learning algorithms is really impressive indeed. But when people are advocating AI research as a step towards some “artificial general intelligence” people (rightly) will raise questions and start poohing you down.
If your plant actually looks like a moss you probably shouldn’t call it a rose. (Even though your moss is actually quite amazing).
I don't understand your analogy here. It really is the machine talking; there is no human hiding behind the curtain.
On the other hand parrots can act in the environment, LMs are isolated from the world and society. So a parrot has a chance to test its ideas out, but LMs don't.
This hurt and will continue to hurt the ML field.
Meanwhile, the chorus of "look this AI still makes dumb mistakes and is not AGI" takes has gotten louder in many circles than the marketing drumbeat. It risks drowning out actual progress and persuading sensitive researchers to ignore meaningful ML results, which will result in a less representative ML community going forward.
With absolutely no evidence (as none can be had about the future) I believe that AI can be reached with computers and programming languages as different from the current ones as rockets are to horses.
Certainly a "real scientist" in every sense.
That’s the only hypothesis that predicts something like AGI from emerging from even bigger language models.
Aaronson clearly isn’t expecting this tech to lead to AGI but that doesn’t make it not important.
Update: See my reply to dudeinjapan's response - I had not been completely clear here.
Ironically, I had just replaced 'test cases' with 'tests', because I thought that the former might seem too generic, and arguably satisfiable merely by rephrasing the problem statement as a test case to be satisfied.
Having said that, your point is kind-of what I was getting at here, though in a way that was probably way too tongue-in-cheek for its own good: When we consider all the activities that go into writing a program, the parts that AlphaCode does not have to do are not trivial. Being given solved test cases is what allows it to succeed (sometimes) with an approach that involves producing a very large number of mostly-wrong candidates, and searching through them for the few that seem to work.
https://en.wikipedia.org/wiki/Coevolution
I am not sure I would like to see dogs convolved with humans.
to go meta, I don’t believe that our way of approaching AI in general is going to get us to AGI.
Why? Assume for a second that we could build something that could solve ANY problem. What would happen next? What does an intelligent human with zero emotions do? Nothing.
Fear of dying and the hardwired desire to pass on our genes (or to contribute something to society - which is just another way of shooting for immortality) is what drives us.
I predict that true AGI cannot come before a machine that has feelings, that is sentient.
We already have machines that do that. They’re called humans. It’s very arrogant of us thinking we can recreate in silicon what nature has evolved over eons. Not saying it’s not gonna happen - but not with the current approaches.
Also, a dog that speaks mediocre english? please. this is insulting to dogs and shows out tendency to anthropomorphize everything.
the game changer is not going to be AGI, it’s going to be human/machine symbiosis
but then again, its approach is so vastly different that it makes it really incomparable with our typical modus operandi. It's throwing millions of monkeys into millions of typewriters and actually producing "to be or not to be" reliably.
I don't see AI writing Shakespeare in my lifetime though (I hope to die rather soon).
Code is not a goal. It's a tool. The tool can get better. But it will always be a tool and someone has to control the tool.
Writing code is super easy. It's the easiest part of our job.
The hard part of our jobs is to understand why we need to write code and right along that, what we write code for ?
Maintenance Structuring Domain knowledge Refactoring
Those are hard things to do. And they are not solved by having a program writing code for you.
Yep. Some efficiency gain is surely possible with "intelligent" autocomplete (i.e. codex instead of intellisense), but it must be code you wanted to write.
"If you think you're paying me for typing code, maybe you should have hired a typist and not a programmer?"
I think with current generation of algos we can get to very good google searches but ones that you can throw own documents/code at. Later probably also DB schema etc.
In another thread I predicted we’d see competitive programming “solved” in ten years or less.
I didn’t rigorously explain what I meant by that in that thread, so I’ll clarify what I meant.
I expect AI to beat humans at competitive programming at the same rate as AlphaGo beats human Go players.
There could be diminshing returns soon but I don’t see what would cause them.
My guess is that's where the difficulties will arise.
Give the AI as much data as it wants, if it manages to solve something, it means it's a problem worth automating (is this a cat?)
Actually I doubt it. There are different ways to learn, the cheapest of which is observation indeed. There's also trial and error, how most enthusiast start with little more than a description of the grammar if it's primitive enough, famously so for the BASIC varieties. Of course this leads to local minima and frustration quickly, Turing Tarpits e.g., but it also leads to independent discovery and rediscovery. At the next step of learning, I believe, it's the differentiation that makes ... all the difference – ie. learning about the development of languages and eventually of computing machines per se. At a higher order of learning it cannot be limited to calculation without a semantic interface (and vice versa).
I mean, etymologies like ominous calx (Latin "pebble", cp. chalk, calcium) for calculation, and symbolical notation show the kind of "code" that we copy from. That said, I'd think at the current state of the art, the ai would have to look at the machine code as the representation of its output eventually (perhaps because I found myself wanting to do that), even if encoded through macros and programming language devices, which it could do entirely unsupervised.
However, understanding the task is a more general problem, isn't it?
That would be cheating.
Are they already doing that? If not expect 2x improvement easily.
Also it would be great if algos were aware where in embedding space their solution is and then try to jump between "approaches".
We're getting close to an era where the drudgery in maths can be automated way.
Has anyone tried searching for new basic operations, below the level of neural networks? We've been using these methods for years, and I doubt the first major breakthrough in ML is the most optimal method possible.
Consider the extreme case of searching over all mathematical operations to see if something really novel can be discovered.
How feasible would this be?
Isn’t that a truism? “Understanding” is equivalent to AGI. Nobody would argue that we have AGI yet, but the intelligence-sans-understanding is somewhat similar to animal intelligence, which was the precursor to human intelligence. What should scare us is that we know that animal to human was not a difficult step evolutionarily.
Do we know that? If that is not difficult, why is it so rare?
All of which is to say that memorization as you describe it seems like a plausible path to AGI, and humans don’t deal with weird inputs well either. It’s not like we are trying to make the ultimate single intelligence that rules humanity (although that may eventually come to pass). But something of roughly human competence certainly seems achievable.
If you believe that people are "general intelligence" systems, then your comment doesn't imply that the existing artificial systems won't achieve general intelligence, because people fail massively on weird inputs.
Consider optical illusions:
https://newatlas.com/science/best-optical-illusions-year-202...
Consider audio illusions:
https://www.youtube.com/watch?v=yKR2pGwavqE
Consider tactile illusions:
https://www.youtube.com/watch?v=WdhnEq76_PA
* Machines fail on weird inputs, therefore they will never be intelligent;
* People fail on weird inputs, therefore they cannot be intelligent.
Any moonshots working on this yet?
To clarify, "one million" is the number of programs generated by AlphaCode on the CodeContests dataset, not the CodeForces one (although the former is a superset of the latter). The results on CodeContests are reported in table 5 (page 15 of the pdf) of the DeepMind preprint [1]. The results on CodeForces are reported in table 4 of the preprint (page 14 of the pdf).
I can't find where the paper lists the number of samples drawn for the CodeForces results, but on Section 4.4 the preprint says:
Sampling from transformer models can be easily parallelized, which allowed us to scale to millions of samples per problem (...).
Note the plural. "Millions" can mean one million, or a hundred million, but, as far as I can tell, the preprint never makes it clear which it is. Note that the results on CodeForces (table 4) are averaged over 3 evaluations in each of which AlphaCode generated "millions" of samples and finally submitted 10. So it's 3 times "millions" of samples for the actual results in table 4 (CodeForces). I assume those were the top 3 of all evaluations.
The idea that the lines of code in a target program are related to the cardinality of the program space that must be searched before a target program can be found is not completely unfounded. For sure, the cardinality of the search space for programs is some function of the number of tokens that must be combined to form each program in that space. An _exponential_ function because we're talking about combinations of tokens (assuming only grammatical strings are generated it gets a bit better, but not by much). We can take lines-of-code as a rough proxy of number of tokens, and in any case it's clear that the cardinality of the set of one-line programs is less than the cardinality of the set of two-line programs, and so on.
I'm not sure if the "Update" bit above is claiming, that AlphaCode must generate the _entire_ program space of _all_ k-line programs before it can find a solution to that problem. To be honest, even I don't think that AlphaCode is _that_ bad. Having to go through the entire program space to find one program is the worst case. On the other hand, AlphaCode does perform very poorly so who knows?
Regarding talking dogs, I have never met any, but I have heard of a horse that can do arithmetic [2].
__________
[1] https://storage.googleapis.com/deepmind-media/AlphaCode/comp...
We’ll have mostly automated local factories producing whatever you need and be able to sell it for untraceable crypto.
I want that to happen because it will enable resistance movements against the misaligned AI that will inevitably emerge.
I model misaligned AI as like an imperialist state. It’s impossible for a single power to conquer the world because every other state allies against it. That will be how AI alignment is solved.