I'm with you; I think ebooks should be distributed in some variant of XeLaTeX and rendered using its toolchain. You retain the structure and can change the size of the text, font, etc. as the user desires. It would be leaps and bounds better than the current level of typography you see on the iPad and Kindle.

The only thing I can think of is that normal people, even book-lovers, don't care. We have been trained by reading webpages so much of the time that we have low standards. But the sad thing is, ebooks generally have worse typography than your average blog post read in a typical web browser. I'm looking at you, iBooks. Jesus.