"can't tell apart" isn't a transitive relation. That is, just because I can't tell A from B and I can't tell B from C doesn't mean I can't tell A from C.
So, if no two indistinguishable colors have distinct names, and for every pair of colors have a path between them of colors such that adjacent colors in the path are indistinguishable, then all colors would have the same name.
The solution of “assign a degree of applicability if each name to each color” and allow a color name to have different levels of applicability for a pair of indistinguishable colors, sorta solves the problem?
But, in a sense, isn’t “to what degree do each of these names apply to this color” just a kind of identifier like a name is? (Though it has the advantage that we can talk about the identifiers being very close to each other )
We can’t have identifiers for colors always be the same iff the colors are indistinguishable, because “is the same” is transitive while “is visually indistinguishable” is not.
Therefore, in order to be able to describe a large variety of colors (with say, rgb), we choose to use an identifying scheme (such as rgb) which has different identifiers for colors which are visually indistinguishable.