I agree with you 100%. Trouble brews when people are illiterate, but unless you're pursuing a literary career, there isn't too much use for reading fiction. Nonfiction has its own benefits that attract anyone who wants to learn in general, or has complex goals.
Remember the phrase 'bookworm'? It wasn't always a cute alternative to the word nerd. I know about it, because I wasted years of my life reading crap fantasy novels, hardy boys casefiles, etc.