In America, Black people as a whole are a distinct cultural group with their own foods, words, holidays, mannerisms, and media. The are in the special case where they don't identify with their ancestral lands because all culture pertaining to that was forcibly stripped away.
White people, at least in America, aren't a group with shared culture. The only thing that really unites them is the color of their skin. White people also tend to have an actual ancestral culture they identify with highly-- usually the culture of a European country their predecessors emigrated from. And as far as I know the vast majority of them don't want to identify as 'white', as they value their ancestral culture more.