You would imagine this to be the case, but I've definitely been asked to serve as a reference by people who should have no rational reason to think I'd be able to speak positively about their employment. In those situations, I've always declined, indicating I was not comfortable and suggested that they use HR instead to validate their dates of employment here.