I would argue his way of thinking of numbers makes him slower at doing calculations.
When you create a 2D visual representation of a number system you want to choose a shape that has the same properties as numbers. Namely the shape must be monoidal under composition. This allows you to keep one type of shape
For example (int + int = int). When you compose two triangles together you get a parallelogram, so triangles are actually kind of bad as you would need to classify several different types as numbers. (triangle + triangle = parallelogram) The only shape that I can think of that is monoidal under arithmetic composition is rectangular quadrilaterals with at least two parallel sides.
Examples: Rectangles, parallelograms, and trapezoids each can be composed to form another shape in its own class. With rectangles likely being the most efficient representation as they are fully symmetrical (to compose two trapezoids to form a new trapezoid one trapezoid has to be inverted, this does not happen with rectangles).
So his even number visual representation is quite good (it uses blocks) but his odd number representation is all over the place and seems arbitrary. Just look at 9. It involves "orange peeling" another number just to shove it into the little dent. His system involves mutating, rotating and changing the shape of each "number" in order to perform composition. This costs more "brainpower" to do and is the main reason why I don't classify his ability as a "gift".
It's highly inefficient. I think many HNers are mistaking it for a super human ability. I don't agree. This is more of an interesting ability then it is a talent.
But that's just a guess. Would actually like to see a quantitative measure of how fast he is at adding numbers under his system. This would definitively answer the question.