Working out hard is additional stress. It boosts cortisol just like job stress. Stress depletes magnesium, and so does hard exercise. You will be hurting your health, not helping it, if you're stressed out and then do crossfit type stuff. That you can't train hard and endure other forms of stress is well known among athletes.
I exercise often and hard because I'm vain, but the health benefits of exercise are way oversold in my opinion. The pyramid of good health is 60% diet, 20% sleep quality and stress, 10% social and spiritual life, and then at most 10% exercise.