Anecdotally, the worst effect I'm personally getting is from my mental health. There is this thought inside me that I should not be doing it and that it's bad for me. That is, psychologically, the worst effect for me, is my own mind punishing me for it. Due to that I have been quitting on and off.