You generally leave the baseload (coal + nuclear) on, and shut down "peaking" capacity, which is, at least in the US, mainly natural gas. Coal is more expensive to build, more expensive to turn on, and cheaper to operate. Nat Gas is cheaper to build, cheaper to turn on, and more expensive to operate