To give a simple example, the question is "Did the Umayyad caliphate bring new irrigation techniques to Spain? What was the overall balance between harm and benefit they brought to Spain's population?", and not "Do the new irrigation techniques justify their colonization of Spain?"
Somehow, we are able to dispassionately examine the harms and benefits when it comes to Islamic-ruled Spain. So why should we look at Britain's empire any differently?