I don't know that gender has much to do, necessarily, with the fact that it's hard to find a partner when your life consists of not doing much and watching reruns.
The guilt-shame-depression-guilt cycle is certainly reinforced in non-working men by indoctrination in a particular strain of patriarchal gender roles. Having a little voice in the back of your head constantly yelling at you for being a failure and "not really a man" can't be healthy.