A very important result from this study is that GPT-4 is in many ways
worse than GPT-3.5, which was worse than GPT-3. More RLHF makes LLMs hide the overt racism better and leads to a false sense of security ("thank God GPT finally stopped using n-bombs") but they are learning more
covert racism from their giant pile of badly curated training data, and this covert bias is not being adequately addressed in RLHF.
Any company using LLMs for hiring decisions needs to be investigated by the feds.