You don't, but it's also not necessary. It's impossible to objectively assess someone's subjective experience, the best we can do is look at groups of people and attempt to find reliable indicators.
The point is that some people will over-emphasize any given trait, and others will under-emphasize it, so on average it evens out.
Think of color perception for a similar conundrum. How can you be sure that the red you see is the same as everybody else is seeing?