Maybe the virus was not a pandemic 20k years ago. Small tribes that carried genes that are resistant to the coronaviruses spread later on as they established larger communities/cities.