I completed Spanish, Portuguese, and Chinese on Duolingo (as in level 5/gold completed) and feel like I know the equivalent of at best a 3-month one-hour-a-week type entry level class.

The stuff that works better for me are actually online classes (for example Coursera has 6-part HSK Mandarin classes) since they are well structured, they have lots of video with natives speaking, and they deal with relatively useful topics. Duolingo taught me to say things like "the book was eaten by the yellow horse" before I could even ask someone how old they were.

That said, I feel like they've been going in the right direction by adding things like duolingo stories. If they expand those and add more interactive stuff it could get more interesting.