Growing up far away from the US, but still around western-ish culture, my experience was that overpopulation fears were primarily about resource shortage and creation of large poverty class within the same society; more recently, in context of immigration, about culture shifting to something that's alien or hostile to people living today (that's the "${immigrant group} isn't assimilating" complaint), and about housing and unemployment issues. Underpopulation, in the meantime, I see primarily talked as risk of there being not enough young people to fund old people's retirement pensions. It's a major factor why people I know in my generation all assume we won't get government-run retirement when we get old; the system will collapse due to negative population growth.
Might be a perspective of someone from the old continent, but those issues are rarely if at all viewed through a racial/ethnical perspective (immigration notwithstanding) - even the feared poverty class is just expanding within the same society.