That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement learning, expert systems (another candidate for AGI in the 90s), soft-optimization. Each of those tools are better than GPT for some problems.