Don't forget that the Apple I was basically a glorified video terminal, so the text mode came first. Also in the Apple II. That's the reason there are 7 pixels horizontally per character.
The graphics mode is just a juiced up text mode, so the 7 pixels were kept.
And for the video shift register, it's no difference if the input comes from the character ROM, or video RAM.