What is wrong with the CodeGen model that they are using?
It is a reasonably large model (up to 16B params) that has already been trained on both natural language and code. I would expect it to underperform larger models, including GPT-3.5 and GPT-4, but this should still be very useful for autocomplete and boilerplate in simpler cases. It is a bit under trained compared to Chinchilla, T5, or LLaMA, but it still performs well.
According to the paper[1], this model is competitive with the largest Codex model that was the basis for Copilot originally.