Look up AI safety and THE aligment problem.
This isnt a matter of training data quality.
This isnt a matter of training data quality.
If we could align an AI with incentives in the same way we can a person then youd have a point.
So far alignment research is hitting dead ends no matter what fake incentives we try to feed an AI.