I do think it might be the case if you were white. Why? Because I am, and you just described difficulties I've faced perfectly. I am fairly confrontational - I believe objective reality exists, we can measure and reason about it, and feelings should be ignored. I'm also 6'6", currently a cruiserweight but often a heavyweight. I intuitively think statistically, which I'm told adds to things.

A very good friend of mine told me she is sometimes intimidated by me. We have difficult conversations on a stairwell - she stands 2-3 steps up.

In my next job, I'm going to be very careful to work somewhere that shares my style. In many cultures (e.g. the NY hipster culture) I would be better off hiding the real me. So I strongly suspect this may just be you (and me), not a race thing.

Of course, human perception is biased and irrational, so my experience may be 100% me while yours is 50% you 50% race.