In over two decades of work in corporate as well as entrepreneurial environments, I've not seen a difference in efficacy within groups that would be attributed to being either racially diverse or non-diverse.
For example, if there were three groups working on a given technical project, one composed of all white men, one composed of all Asian women, and one composed of a mix of race and gender, would the diverse group produce a superior result? My experience is that they would not. Sufficiently and equally incentivized and qualified, all groups would likely produce similar results.
So, I'd consider myself as somebody who's "against racial diversity" mainly because I haven't experienced it improving the core competencies of my company -- which is what I care about. Nor have I found compelling research supporting higher performance by more diverse groups. Thus, efforts to improve diversity, for diversity's sake, in my experience, feels like an effort to make a change that's not related, and might even be a distraction, to making my company more effective.