I agree. Seems like what Feynman really wants to say here is “this isn’t a question I’m personally interested in”, but frames it as “it’s dumb to be interested in this question”.
We are only experiencing the end result of what our senses sample and our brain processes. That’s clearly true. How much that end result diverges from the real thing is difficult to answer, but in a straight forward way you can look at an optical illusion to see that there is some divergence.