I suspect loneliness/isolation is the biggest issue for most.
At the moment it's great because I'm at home with three housemates/friends. But I think I'd get lonely pretty quickly if it was just me. Co-working spaces might solve this a bit (never worked in one so don't know how much people socialise).