We were unfortunately young and in poor company, so the idea didn't receive the appropriate attention.
[1]: https://en.wikipedia.org/wiki/Qualia#Inverted_spectrum_argum...
Blind people surely compensate for the lack of visual information by deliberately eliciting audio and tactile feedback that others don't need.
Also, watching others interact with the world is never the same thing as interacting with the world ourselves, because there's a crucial piece of information missing.
When we decide to act, we know that we just made that decision. We know when we made the decision, why we made it and what we wanted to achieve. We can never know that for sure when we observe others.
We can guess of course, but a lot of that guessing is only possible because we know how we would act and react ourselves. A machine that has never intervened and deliberately elicited feedback cannot even guess properly. It will need incredible amounts of data to learn from correlations alone.