Because you were hitting the power envelope limits in the CPU in these cases too. You might not see the heat, but the CPU cannot carry the power required to keep that core at non-AVX speeds with these power-hungry blocks operated at full speed.
As I said, to add insult to the injury, Intel didn't share the exact details of its AVX implementations and frequency ranges it operates, either.
Ah, publicly sharing your findings is/was forbidden too.