Huh. That's interesting. In a "people's brains think surprisingly differently sometimes and we rarely think about those differences when the resulting behaviors look the same" kinda way.
I mean, I guess that's not so foreign...But, I tend to think of it as pulling out the notes I need from the scale, and not actually counting up to them. e.g. in my brain I'm grabbing the third and the octave (well 7th, if you've got a third and then the fifth of that third, which I guess is why you're preferring 0-based) that are already there...not climbing up them to find there's the tonic there. I mean, I can see that it's a fifth interval if I go from E to B (in C), but unless I'm building a chord on E, I don't care..it's either the 7 in C, and I'm not so much thinking of its relation to E as I play it, and it's a phrase in C, with maybe an Em chord (either implicit or explicit) underneath; or I'm playing jazz, or some other very chord-based music, and I want my phrase to be relative to the chord we're currently playing (so we're inside that Em, and the key is less relevant).
Sight reading is different, as well, in my brain, but, I think it even bypasses the intervals to some degree and is just distances and shapes and an awareness of the key I'm in. I don't read much these days, but I recall it working best (or at least fastest and most accurately) when most of the theory was turned off in my brain and I just let the shape of the notes (their distance from each other) guide me. But, I feel like it's only in improvising and composition where one would be doing any sort of interval math. But, maybe I'm wrong.
When are you doing this kind of math? When reading, improvising, playing memorized pieces, or composing?