This analysis of politics is only coherent if you ignore the existence of ideology entirely. It's true that "facts" don't necessarily unite people of competing ideologies because that's not how ideology works.

You could say a politically charged fact like "People of race A commit crime at 2x the rate of people of race B".

One person might then conclude "We need to put more police in predominantly A neighborhoods". Another might conclude "We should bus kids from A neighborhoods to B neighborhoods to normalize education rates". While yet another might say "Our public education shouldn't be funded primarily via property taxes, which systemically leads to under-education of lower-class individuals".

Same fact, different conclusions. You can't reduce it down to just "allegiance" or "feelings". I'd argue that understanding your ideology is the only way to make sense of politics. Most folks never really stop to examine their ideology, and just end up wherever they've been left by teachers / parents / talking heads.