And, for people who aren't white men, oppression is a lived reality, not an abstract concept that you can attempt to rationalize away. It is both rational and justified to think that prejudice is a predominant factor in virtually everything, because it is.
Finally, oppression isn't "statistical." The people who cause those statistics have names, faces, and addresses. As do the people who act as their apologists. The personal is the political, but the political is also personal.