The mental model of being an offset to a memory address is, I think, part of it.

I'd be surprised if the use of zero vs one actually made any difference, from a number of operations point of view -- from the compiler's point of view, the first element in the array is the first element in the array, no matter what we call it.

I mean in an extreme edge case maybe if you are computing an index, and it happens to be zero, then in your math maybe some identity related to zero could be exploited (go to element simple_integer*complicated_function(), where simple_number might be zero) but that seems a bit silly.