I mostly hear this word in the context of the Romans, but I thought that it was a term for facial hair- implying those external to the "civilized" empire had a tendency for ragged and long facial hair.
Hence the term 'barber' for a person that cuts said facial hair.