Language is just a form, what exactly is encoded inside of the model can be very different. And to encode logical reasoning inside of the weights with activation functions is more than possible.

Models solving IMO level problems imo proves it.

I also think you greatly overestimate human intelligence, the fact we got AGI is nothing but barely side effect of evolution.