I am reminded of an essay by Stephen Jay Gould, on the history of preformationism, and the debate over whether the homunculus resided in the sperm or the egg. The idea that a human could be represented by an ordering of simple chemicals was inconceivable at the time.
The idea that physicalism requires a specific physical locus for redness is no more sound than the idea that genetic theory requires human-ness to be located in a specific base on a DNA molecule. Emergent phenomena are real, common, and not, in general, particularly mysterious (though the mind certainly is, at least to me.)
More prosaically, maybe redness is actually localized, as a memory of a sensation. Is there anything more to redness than a comparison to prior sensations?