I also strain to find really substantive criticism in Fisher's original piece. "Russell Brand as popular left vanguard" has not really aged well, he's done the standard turn from proto-dirtbag-left to exactly the kind of media-talking-head-dumbass the leftists Fisher decries predicted he would.
I am broadly (but weakly) class-first in my leftism, but the kind of purity Fisher is demanding is bad tactics, but also counter-materialist. Race exists orthogonally to class but still has material implications - which is simply to say, black people still get profiled by cops even if they own a factory. For a good analysis of this, see Racecraft: The Soul of Inequality in American Life by Barbara and Karen Fields. Conversely, "the means of production" is not what it used to be - capital operates correlated but not identical to control over information vectors, and owners of those vectors exercise great power over people who can control their own production but not the distribution of it; see McKenzie Wark's Capital is Dead: Is This Something Worse?
The left is traditionally focused on class because Marxism, via materialism, made strong arguments this was the lynchpin of inequity. But if you run that same materialist analysis today, the evidence for class's primary, let alone a sui generis status, is the weakest it has ever been.