I totally agree with you. I have experienced lot of stuff by myself that you mentioned.
I am living here in US from last 7.5 yrs. I still have very thick accent. I tried a lot to get rid of it. Even hired an accent trainer but due to lack of time could not practice. At the end, my trainer gave up. I have lost lot of opportunities due to my accent. It is not just my accent, my tone is also so rough that when I speak my voice becomes so rough and it sounds so unpleasant. I think I need throat surgery to speak properly.
I don't blame them who show me off faces. My only complain is they don't even try talking with me. I can see the change in their body language when they see me coming.
Not all are bad. Some people are still very welcoming. I wish there were more of them.
I have seen my European friends getting so warm welcome even though they speak more worst than me. I am not jealous about this. Just sharing how people treat others differently who are from different origins