>> The guys are absolutely right - it's still early days for this tech and the
sky would seem to be the limit.
It depends on what you mean "early days". If it's about Codex itself, then
that's probably right, it's a new-ish system. However, the task of creating a
computer program by another computer program, variously known as "automatic
programming", "program synthesis", "program induction", "inductive programming"
and so on is not new, rather it goes back to the 1970s and probably earlier
still. Very briefly, these terms describe a constellation of approaches that
automatically create a program according to a specification.
Placing Codex in the context of this earler work, it is an example of program
synthesis system that composes programs from incomplete specifications,
variously given as natural language specifications or what we can call "code
snippets" (i.e. you start writing code and the system completes it; I don't know
the propper term for this kind of specification). As such, it is one in a long
line of systems that predate it and it is not even the most impressive of those
systems (37% accuracy is nothing to write home about).
The reason that you misidentify it as something new, by your "early days" turn
of phrase, is a peculiar tendency of deep learning researchers to ommit any
references to relevant prior work, for reasons that are difficult to discern.
Sometimes the reason seems to be honest-to-god lack of familiarity with other
approaches than deep learning (even other neural networks approaches).
Sometimes it seems to be more of an attempt to claim well-trodden territory as
brand new and trumpet a small step forward (or not even that much forward) as
the breaking of new ground. Sometimes there seems to be an attempt to avoid
unwanted comparisons to different approaches with different trade-offs, that
could make the deep learning system look more lackluster than desired.
In any case, there you have it. Codex is not any kind of breakthrough or
innovation. What is new is the sudden interest in program synthesis. Perhaps
this is a positive thing and program synthesis approaches will finally begin to
be adopted in the industry. But knowing how these things work, and how what gets
hyped and what doesn't is altogether without any rationality, I'd guess probably
not.