A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text
Isn't this the same problem that Github Copilot has?
Fundamentally it has no idea whether code works. It doesn't even know what the problem is.
It just spits out things that are similar to things its seen before, including buggy code from Github repositories.
Not sure why it's so popular. I guess it helps you write status quo code faster (the status quo being buggy and slow) -- I would rather it help us write better code.