Lithium batteries, even when they don't provide power, slowly lose power.
They can also "degrade" when they're not in designated voltage range. This is one reason why most batteries are soldered, because they need to be regulated at all times.
So no, you cannot "turn off" a battery. A battery should either provide power, or be plugged to a charger.