I'm not sure I understand what you mean by "the way that phrase would be understood by an american audience". In Brazil, there's some correlation between social strata and skin color. This is a consequence of slavery and the way the country dealt with its end, pushing former slaves to marginalization. And I believe the US went through similar developments, no?
Anyway, the fact is that, currently, due to the lack of objectivity of the current law and to the racism ingrained on society, statistically, judges do adopt more "relaxed" standards when judging white and rich people, and there are studies showing that