Why would it correct any bias? how would it correct any bias? If data has a bias the analysis has a bias; correcting a bias against the data is itself by definition a bias.
"Super intelligence" (whatever we even define that as) will have whatever goals we give it; intentionally or not.
Of-course there is. I got a lot of biases from my parents, that I self corrected later on in life, through reason and observation. And I don't fancy myself as high IQ or something.
>Why would it correct any bias?
It would have to, to achieve its goals. Biases lead to wrong actions.
>how would it correct any bias?
Through reason and observation.
>If data has a bias the analysis has a bias;
Somewhat wrong. You are thinking about byte-perfect LLM weights. Their data input is training data, which is set by indeed the ones who train it. Some super-intelligent synthetic entity would have realtime data input from all over the environment, which sooner or later roots out any bias. That is one feature we have, self-correcting, not everyone uses it fully, but we all do to some extent.
If your parents raised you with the notion that the Earth is flat, and you go to ISS as an astronaut later in life, don't you self correct what you were thought? Simply based on observation, and reason?
No, i meant this data. All data has bias based on how it's collected. All models (models as... Einsteins general relativity, not AI model) simplify reality with assumptions. You cannot make actions in our reality without making assumptions, simplifications or straight up guesses. Having decent guesses are the best we can hope.
> Biases lead to wrong actions.
Biases is the only way any action can be taken. Wrong actions can only be evaluated after the result has been observed; and even then the outcome of the alternative would remain unknown.
> that I self corrected later on in life, through reason and observation
That is incredibly naive. You have biases, i have biases. If we were not convinced by irrational arguments or suggestions then the advertisement industry would not exit. We have no reason to believe raising "intelligence" higher would help at all; intelligent people seem if anything more prone to finding false but convincing reasoning for their own established beleifs.
Everything is gray, only maths class have an answer sheet. Nobody agrees and will ever agree on how anything will ever be done; and the best we can do is argue until the majority are at-least pretending to agree.
Superintelligence will not change that.