How is an AI supposed to work that out? Given the talk about how these models have a tendency to confabulate, this would seem problematic. Idk though, maybe this is now a thing?
Yep. For instance in this paper[1] they asked an LLM a bunch of political questions to figure out what its political alignment is.
[1] "Discovering Language Model Behaviors with Model-Written Evaluations" https://www-cdn.anthropic.com/e4f69aacd8c0905030172bc6eb480c...