The article you yourself linked says the game's development troubles were from pressure to surpass their previous game which was a smash hit.

How would a better engine fix that?

They didn't need "a better game engine" to create Celeste.