I've managed single man remote teams.
It can work depending on the mindset.
You're right, if the single man team is looking to socialize and feels a lot of FOMO not being in the office then it won't work.
You obviously don't hire a person like that for your single man remote team.
I've found that in the vast majority of cases single man remote teams are remote because they're optimizing for a particular way of life that they enjoy; for example, the country side lifestyle with their partner and kids.
Then it usually works out fine; we will still organize for a on-site every 2-3 months for a week just to give them a bit more of sense of belonging.