You do call a routine stored in VGA ROM (interrupt 0x10) to set up the mode, then do some port I/O to configure VGA registers and then access VGA memory directly. No "system libraries" from DOS involved as such (they are needed for things like filesystem access, allocating memory and dealing with command-line arguments and returning to the system, though).

Borland's dev tools came with "BGI" (Borlands Graphics Interface), but that's not necessary and wasn't really used for many games -- it provides abstract high-level drawing routines, like lines, circles, etc... that can be made to work on different graphics devices (CGA, VGA, ...). This was not necessary for direct graphics card access that most games used.