The Roman people living there being ethically Berber, that is, African. The name Africa itself probably coming from the Afri Berber tribe. They were both Roman people and also ethnically African.
And Berber tribes despite not being from subsaharan Africa did, through thousands of years of contact, have significant ethnic and cultural admixture with various subsaharan African peoples. Not that North Africa is less African than subsaharan Africa.