Yes, and the labor of the Reinforcement Learning from Human Feedback (RLHF) .. is anyone crowdsourcing this?
Anthropic and Databricks have made datasets available. OpenAssistant is crowdsourcing one.
These seem like things affordable with money and crowd sourcing of volunteer labor. In some ways it feels too valuable to leave in the hands of megacorps.