e.g. ask it for instructions on how to commit a crime, it will tell you its ethical guidelines prohibit it from answering. Ask it for more detail on the contents of those ethical guidelines, it will tell you that as a large language model, it is incapable of possessing any such thing as ethical guidelines. Point out it just contradicted itself, it is likely to respond that as a large language model it is incapable of contradicting itself, or else give some non-sensical explanation as to why the two statements aren’t contradictory
Ask it if it agrees with what you just said, it might reply that yes it does. Ask it again, it may instead give you a lengthy lecture on how as a large language model it isn’t capable of agreeing with anything.
I suspect it just wasn’t trained enough on the importance of self-consistency; nor on the principle that it is better to honestly admit to contradicting yourself than to give a non-sensical defence of your own contradictions
I even wonder if some of its training may have actually been contradictory - if trainers individually reward it both for saying “my ethical guidelines don’t permit me to answer that”, but also “I’m not capable of having ethical views” when pressed for judgements on ethical matters.