I don't think anyone is claiming that the latter is widespread, and I also don't think you're acknowledging that there are many people who can't do the former at all (myself included).
If you ask me to imagine something, there is no visual component to that at all for me. Internally, externally, nowhere.
Moving the goalposts is bad arguing. Bad arguing is often due to bad-faith arguing; but good arguing is hard, so it's also often accidental. Yet, people equate pointing out flaws in an argument with accusations of intellectual dishonesty: it's been politicised.
[1]: https://www.lesswrong.com/posts/fEX7G2N7CtmZQ3eB5/simulacra-...
[2]: https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-...
Sorry, but this is too funny. There's some sort of fundamental difficulty when humans are trying to compare their varying (or not?) qualia.
Like those stoner-philosphy questions. ("Is the 'red' I see the same hue as the 'red' you see? How would we ever know??")
There's an element of that unsolvable problem here; of knowing whether the other person is describing their internal mental state using the same definitions as you are.