Now we have a machine that is inconsistent and messy (and helpful!) and nobody seems to know what to think. Maybe we stop applying machine notions to this sort of machine? Stop expecting certain, consistant output. Understand that it's sometimes messy. We have these expectations already when working with humans.
Humans have biases, if you ask a human a loaded question you can expect a loaded response. If you train a LMM on a dataset that contains those human biases, why should you expect the result to be anything other than similarly biased?