Accessing beyond bounds is undefined, but leaving pointer arithmetic outside of objects undefined would preclude a lot. For example, building an OS page table or DMA, or RDMA, or MMIO, etc...
When an expression that has integer type is added to or subtracted from a pointer, the result has the type of the pointer operand. [...] If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined.
[1]: http://stackoverflow.com/questions/18186987/decrementing-a-p...
I believe that C99 added the ability to losslessly cast from a pointer to a uintptr_t and back again, but, IIRC, the compiler didn't have to support this in C89.