There's a difference between a lack of affection/physicality because you're depressed and because you're using an SSRI.

I still wanted to be close to people while I was depressed - I just felt useless and unable to do much else. On the SSRI I lost the desire to be affectionate, both around others and when I was by myself. It took around 6-9 months after I came off the SSRI for me to feel like I returned back to a normal state.