Down sides? Choice. (You want a different translation of The Brother's Karamazov? Ha!) You still need a hand, or at least a finger, to pause or go back, like when your mind inevitably wanders. You can't share an audio book with a friend, a downside of e-books too. And then there's the whole mind-wandering thing. I can't multitask. I can't listen to an audio book at just any time I need my hands for something else. The context switch is a buzzkill.
The key to a good audio book, in addition to the book itself being good, is a good reader, so here are some of my current faves with splendid readers: A Clockwork Orange narrated by Tom Hollander; Lincoln in the Bardo, Nick Offerman; Mrs Dalloway, Juliet Stevenson; Lolita, Jeremy Irons; and Milkman, Brid Brennan.