Nope. If I have to put my energy and resources into feeding and caring for myself and my family, then my "IQ" - my ability and experience in engaging with abstract data - suffers.
Yes, there will still be an achievement gap - that the achievement gap currently reflects socioeconomic factors is unfair.