Any time an absolute reference point is assumed, all downstream calculations are effectively absolutes
You can also ask a UI designer about "absolute" colors. One color needs to be different depending on the context it's in order to LOOK the same (it's size/thickness, background, what it's next to).
If OP had said “perceived color”, I would agree. But we’re talking at pixel level here.
Suppose you have a reference pixel but don’t know its value. You don’t get the color right but there is plenty of other information there, right? Which pixel also probably limits the range of values since your highest and lowest values are probably within a certain range.