It is presented with a tone of "music theory, the way a musician could never understand it, because they do not know what a sin wave is" which makes it so infuriating.
there is a big difference between seeing that pairs of input and output are correct, and knowing that the system is correct. there are invinitely many in and output pairs and only checking some while you vibe code your tool is never going to be a reliable method
why would you support a person in the sad psychosis, that they can ai their way to a solution of cancer, and that they can do so better than medical professional teams and research institutions at universities wth lots of expertise and lots of resources
Tesla went from "a cool tech company, and also the ceo does rocket stuff, which is kinda nice" to "what is wrong with that f*cking nazi jerk" in the last 10 years in my circles, and it is very much deserved.
It is of course because algorithms can be repaired when they are buggy, but a large language model can not, because it is impossible to look at its weights and say, look, this is where the mistakes has happened.