FYI: I've put together a free book format called Manuscripts. See http://manuscripts.github.io and find an example bookshelf @ http://yukimotopress.github.io Happy book writing. Happy book reading.
PS: That's also the direction the World Wide Web Consortium (W3C) tries to go (but failed). See the upcoming https://www.w3.org/TR/wpub/ recommendation / standard / spec trying to replace EPUB. Compare the book manifest of manuscripts with the book manifest of wpub to see what format keeps it simple.
PPS: And to keep apple and oranges apart - FYI: pandoc is NOT a book format (it's a text processor), thus, you should compare Manuscripts to EPUB or the new WPUB.
But until my Kindles and Kobos can read it, I would never use it to consume content.