Standard mirrors are only ~98% reflective (https://en.wikipedia.org/wiki/Mirror#Reflectivity). For example, "infinity mirrors" will fade as the number of reflections increase (https://physics.stackexchange.com/questions/305329/are-infin...).

I initially thought the same, but my current model is that the mirror would also have a very low emissivity so you can't passively cool (only not heat so much).