Growing up, I was formerly a member of a high demand religion, in an area where it had minor representation. I took a lot of flak for that as a kid -- targeted and environmental. Fights, physical abuse, verbal harassment, hostile work environment and more were common. I can empathize with your plight, having experienced it similarly.
That said, I have the "right" color of skin and didn't have nearly as much issue once I removed myself from the area I grew up in (and, later, contravened that religion by exiting and keeping my worldviews private).
I completely acknowledge racial privilege can exist, and that I am a beneficiary of the setup as such, much like the dominant religions in my childhood hometown had religious privilege and the "good ole boys" network is built to serve their adherents locally.
It SUCKS to be the target of negative interaction, it sucks 100x worse for it to be environmental.
My take on the entire state of race relations is that the median person wants interruption of the "good ole boys" entrenchment. This does mean privilege gets reduced for some, which feels like persecution and evokes fear like we see from folks rallying against "CRT in schools" (though it's typically just general history being railed against).
Then we see disconnected takes like PG's "Heresy" recently, which takes the extremist positions on both ends of the horsehose as representative of the median.
I learned pretty early on that most people are fairly banal in what they want -- food to eat, roof overhead, available recreation, build a nest egg and something resembling a family. These things are at risk these days due to class division and economic uncertainty; people in the well-to-do side of things are perfectly happy to see the stress taken off themselves and projected into racial and ethnic divisions because they can afford something like security provisions.
Apologies for rambling, your comments just elicited a lot of things I don't think about all that often.