For the people wondering why they don't see this setting, it has a karma threshold. I don't remember what it is, but it's something triple-digits.
I suspect they are not sorted by popularity. The user setting is "topcolor" and this page is "topcolors", so don't assume that the "top" is implying most used.
I remember seeing that at some point in the settings but I don't think it is there anymore?