Is it as simple as people don't want to get speeding tickets? Certainly any stand out colors increase your chances of being noticed and tracked by law enforcement everything else equal.
There are many people who will not buy a colorful car. There are very few people who won't buy a car if the color is too plain. Therefore people value the colorful cars less because there's a smaller customer base and the market shifts to producing vehicles in plain, minimally offensive colors.