That being said, the caveat "on average" must always be added. On average, on average, on average. Same goes with anything about average tendencies of different genders. On average, on average, on average, averages across large groups do not map to individuals in that group. Emboss that on a mallet and bash people over the head with it until they get it.
I do wonder if there might be some conservation at play in some individual cases -- which kinda matches what you said. There is only so much brain tissue. If you're hyper-good at some narrow kind of analytical intelligence (coding!) maybe some of that brain tissue is borrowed from other things it might be doing like understanding other meat sacks.