Making a mathematical book in mobi/epub is a difficult problem but I'm surprised the publishers weren't aware of the difficulties.

Simple statements can be expressed in pure Unicode (though the Kindle fonts have incomplete Unicode support for the relevant blocks). Anything more than that has to be an image, which is the bane of digital publishing, since it won't scale with the base font size. epub 3 might be able to cope with it but apart from iBooks I don't know anyone else using it.