Can only answer for myself, but yes—I am completely unable to picture my parents.
I'm not completely aphantasic, but the way you phrase the rhetorical contains exactly the massive divide that exists in experience. You believe that such an act is obviously easy, whereas for myself, it is difficult and less an image than a fleeting impression that is more conceptual than it is visual.