I like the paper from Anthropic on Constitutional AI that helps with transparency on how a model is trained, and therefore what values it might have.

https://www.anthropic.com/news/claudes-constitution

https://arxiv.org/abs/2212.08073