Based on my understanding from the recent TED talk [0] they mention that GPT-4 can for example add 40 digit numbers perfectly, but makes mistakes on adding a 40 digit and a 35 digit number. It hasn’t worked out a generic understanding of arithmetic but several smaller case specific ones which may not always be correct.