That's what I've always said to my friends, but they insist on learning grammar, which of course they'll forget after a few hours/days. Three years ago I could hardly read anything non-technical in English, now I can understand Lolita - which is a hard novel to read for a non-native. All thanks to audiobooks.
I found that audiobooks are the most fantastic way of learning a new language... I couldn't have possibly read Lolita or Silmarillion before - They're just too hard for someone who is trying to learn a new language. Long sentences full of new/invented words - It's easy to lose the thread. But listening to a skillful reader reading them aloud, you can understand even the most complicated words and sentences just from the context, and the reader's tone and emphasize...
If you want to learn a new language, do yourself a favor and listen to some audiobooks. Pick a book you've already read in your native language (preferably more than once) and you'll be amazed how easy it is to understand and learn new words (you must have a basic understanding of that language of course).