How do you "debias" an LLM? There is no unbiased standard to test against.
PS: Yes I know CrowsPairs is a dataset with a bunch of flaws. My SO is working, in a team of 10+ linguists and researchers to develop a multi-lingual, generalized version of it which also addresses multiple problems with it. Unpublished work, for now.
[1] https://github.com/nyu-mll/crows-pairs/ [2] https://arxiv.org/abs/2004.09456 [3] https://arxiv.org/pdf/2204.09591.pdf