I mean, think about NIMBYism (or any of the examples above where access to some economic benefit are restricted). White folks have most of the assets; NIMBYism means the world's most vibrant cities become more and more off-limits to non-white folks, who then become even less likely to hold and grow any real assets. I'd call that racist, but again, it doesn't really matter if the people who benefit from racism are Racists with a capital R. They may not hold racist beliefs, and yet they can still benefit from and promote racist systems. We're all complicit in these systems, because they persist today. Denying that, I think, is a way to keep racism working.
How do you fix the real root problem without naming racism and the people who benefit from it as in some way complicit in racism? I don't know. "Racist" is a high bar, and it lets us ignore that so many of us are responsible for the persistence of racism.