True. But you're probably exerting yourself too hard if a workout session taxes you that much (or you have it built up in your mind so you'll hate it no matter what). Remember Worf's exercise of holding 13-pound weights straight in front of him? To do that, you don't even start with weights, you just hold your arms out. There's no shame (literally none, this is actually the secret of developing an exercise mentality/physiology) in lifting incredibly light things, even just your arms.
I do hear you, willpower is precious. But since exercise helps create willpower, too, and stress mitigation, it's a highly lucrative investment.