I wish people would stop looking so much at completion scores and not judge the concept as a failure just because they are so low.
But in MOOC land I get nothing. I did the complete automata class mostly to see if there's much new since I did a brick and mortar version some decades ago, and the "reward" was not exactly fulfilling. I'm not expecting Ullman to fly out and shake my hand, or my phone to ring off the hook from recruiters seeing I jumped thru some hoops, but something more would have been nice? Actually, anything would have been nice?
Since the grind factor was aggressively burned out of me, I haven't completed a single course. I did watch the videos and did the assignments for computational neuroscience and one of the multi-class algo series and the functional programming and the famous AI class.
There's a lot of talk about how grinders don't grind therefore grinders suck or grinding sucks. However what really sucks is the playground presented to the grinders. Its not their fault they aren't being given something worth grinding.
By analogy most MMORPGs that tanked and died, tanked and died because the grind experience sucked. Its not the fault of the grinders that the businessmen couldn't or wouldn't present something as enjoyably grindable as WoW or Eve.
To some extent the whole market segment of MOOCs smells like a freshmen psych class lab experiment, "what happens to grind performance when you leave all the annoyance factors of grinds in place but remove all the rewards, with the hypothesis that almost every rat in the box stops grinding"
And, I expect, that if you start from the assumptions that 1.) MOOC certificates aren't worth anything as a real-world "vocational meal ticket" beyond the value of the underlying education and 2.) Companies offering MOOCs are not going to become unicorns and probably won't even return investors money-- you probably end up with something that isn't designed to mirror a university course.
I'm proposing that as a bottom up narrative, they continue to include extensive grind barriers such as peculiar scheduling and pacing, someone else designing arbitrary barriers in terms of assignments with firm due dates, etc. Yet the reward has been cut out as per reasons in the top down narrative, etc. So from the bottom up perspective you're left with a stereotypical grind game where the reward has been removed, and from that theoretical model, the dropout rate makes sense. Then when people sign up and get something out of partial participation, it makes no sense to use a successful grind game metric measurement when its a failed grind game therefore the players aren't cooperating because they feel no need to do so. Because "MOOC as a grind game" has failed, a MOOC is a place to learn, not a place to jump thru hoops or not get a reward other than making someone else's completion metric result look good.
You can run a grind game without a vocational meal ticket reward model. WoW and Eve do just fine.
My suggestion would be to use advanced technology to abandon the grind game aspects, assuming higher completion rates are inherently valuable. Even if completion rates are not valuable, removal of pointless grind will result in better operating conditions for the students. Why must class schedules correspond vaguely to the northern hemisphere agricultural growing season? If you're not teaching project management, why are due dates so important? Why is there (typically) only one very fixed learning track if more exist or there is a "chose two from column B and three from column C" aspect inherent in the field itself? All of that would be very difficult to implement at Harvard in 1640, but due to technological advances would not be very challenging to implement today.
Of course, as you suggest, there are potentially ways to gamify and provide incentives for learning that don't as closely mimic traditional course structures.
All that matters is whether the user found some useful information in it, and just fully watching one video lecture should be enough for that metric.
So you either saw different approaches to the same problem (e.g. Data Analysis) or everyone discussing a different problem (An Introduction to Operations Management). I am sure that I got a lot (in terms of learning) from that part of the course.
If everyone just picks and chooses some papers/videos and drops out before the end you don't get anyone to grade your work and you have no other work to grade (and possibly learn from).
EDIT: Found a couple of articles:
http://er.educause.edu/articles/2014/12/mooc-completion-and-...
http://er.educause.edu/articles/2013/6/retention-and-intenti...