Its gotta be the shape of our eye, for the same reason the effect works in a mechanical DSLR camera, where light is only refracted through the lens.
The sensor on the back of a camera is a flat plane, which is why (with most lenses) you get an image where straight lines appear straight. The sensor on the back of your eye (the retina) is curved.