I did that. I took the initial test for Japanese, where I answered correctly for sentences I already understood or where I was only missing a single word I could guess by elimination. Given multiple options I couldn't distinguish, I messed up as it should be.
Based on the test results, I was skipped past 23 of the 40 topics I can see in the course. Considering that I've been learning for only three months, that makes the course look a bit short.
When I then tried the next lesson, I did notice some questions repeating, but there was also some variety. So I don't know whether it's possible to just memorize all the sentences without learning grammar.
Based on me trying Duolingo a single time now, I'd now say that the biggest problem is the small amount of topics covered in a course, which limits the vocabulary you know how to use after completing it.
But that's not what the original complaint said, so maybe someone who's used Duolingo 5 minutes a day for several months is still in a better position to explain what the actual problem with Duolingo is.