I added a reply to the parent of your comment with a link to an article I found fascinating about the strange world of labeling and RLHF -- this really interesting article from The Verge 2 years ago:
https://www.theverge.com/features/23764584/ai-artificial-int...