Population density was extremely low though. So imagine it was tough to form large communities and most people mainly only had contact with people within their group and maybe a few others living close by.

From that perspective I find it hard to imagine that neanderthals would be treated that much differently than other outsiders. Well unless they behaved significantly than homo sapiens when interacting with other people..

maybe not the most ethical and objective analogy but dogs and wolves within the same pack seem to get along much better than they do with wolves from other packs. So I’m not sure species would be a huge issue in very simple communities

As you're probably aware, race doesn't actually exist. There is no basis in science for it. It only appeared as a categorizing concept in the late 16th century. Racism is far older. Subcategories such as antisemitism are ancient, but there was no concept of it and no word for it, and we only understand it as such now looking back.