I'm pretty sure Borland used to separate out the IDE executables from the compilers and a couple of other tools tool. I don't have a copy to hand to prove this but I'm sure I used to occasionally invoke Turbo Pascal's compiler from the command line outside of the IDE (due to it being a separate .exe / .com) and I vaguely recall Turbo C++ having a similar design.
I also don't recall build times being that much faster then than they are now. But maybe that's more a symptom of myself compiling on budget hardware previously where as I can now afford better spec'ed dev machines (compared to the market average).