Yes, but the average temperature in vast majority of the areas is higher than the avg temperature of say Europe. When we say Africa is warmer, it is a simplification simply by the virtue that its centroid closer to equator than Europe. Thus, we can say that it's warmer.
Side note: don't use rhetorical questions to respond to someone. It's demeaning. Rephrase "Why? Africa is a huge continent with all kinds of weather."