Yes, that's the point: the numbers are used as both indices and offsets. To any musician or music student, moving from G to B is "going up a (major) third", even though it's obviously going up by 2 notes. The name of that offset ("interval" in music speak) is "third", even though it has a distance of 2 notes.

My point (more generally) is that even though it looks reasonable to make indices start from 1 and offsets from 0, in practice these things can get mixed together. It's not reasonable to get people to use two different numbers for what they see as the same thing (because their use got mixed).