When I've watched people use terminals on Twitch it feels there's big periods of times spent swapping around everywhere and a whole lot of mucking around. Watching people coding real things and dealing with the various normal problems is a pretty good test of an environment, I've never seen anyone do it in compelling way where I didn't think it would be faster in an IDE. That doesn't mean it can't be done, I just haven't seen it and I've been coding for decades and have a bit of an addiction to watching people code online!
There are, however, many times where it's better to go into a terminal rather than try and get things done within the IDE, but most IDEs offer the terminal as part of the experience.
The only downfall of some IDEs is they have little corners of the IDE where driving them through the keyboard isn't the best experience or not fully implemented.