The Most Thorough Summary (to date) of MOOC Completion Rates

How many times have you heard the statement that ‘MOOCs have a completion rate of 10%’ or ‘MOOCs have a completion rate of less than 10%’? The meme seems to have developed a life of its own, but try to research the original claim and you might find a bunch of circular references or anecdotes of one or two courses. Will the 10% meme hold up once we get more data?

While researching this question for an upcoming post, I found an excellent resource put together by Katy Jordan, a graduate student at The Open University of the UK. In a blog post from Feb 13, 2013, Katy described a new effort of hers to synthesize MOOC completion rate data – from xMOOCs in particular and mostly from Coursera.

Via a combination of thinking about ‘what makes a successful MOOC?’, and looking for a topic for my final project on the Infographics MOOC, I decided to try to pull together the various statistics floating around online about MOOC completion rates. I’m trying to see if any differences emerge on the basis of platform or the assessment methods used.

My draft graph synthesising everything I’ve found so far can be found here: http://www.katyjordan.com/MOOCproject.html

What Katy has done so far is compile completion rate data from 24 MOOCs – 19 from Coursera, 3 from edX, 1 from Udacity and 1 from MITx (a precursor to edX). The data is available in a table format or in a graph.

table

 

As you can see on the right side, you can filter by MOOC provider, University or Assessment Type (auto grading, peer grading or some combination).

In the graph view, you can see completion rates vs. total enrollment of each MOOC (the colors represent different Assessment Types).

graph

 

By clicking on each dot, it highlights the MOOC, basic data and link to data source.

graph 2

 

This is the most thorough summary of MOOC completion rate data that I have seen – for xMOOCs. Kudos to Katy for putting together this data, and please comment on her blog post to let her know of new data available.

Some notes based on the data available:

  • The average completion rate of xMOOCs is 7.6%, with a minimum of 0.67% and a maximum of 19.2%. The 19.2% appears to be an outlier from Ecole Polytechnique Fédérale de Lausanne, although it may be worth figuring out how they got their rate so high.
  • Does it bother anyone that we get this data from a graduate student from The Open University but not from any of the xMOOC providers who claim the power of data analytics in their platforms?
  • In the end, the meme of 10% MOOC completion rates is not too far off, at least for Coursera courses.

The problem I see is that scalar metrics of # of students and completion rates are insufficient for open courses, where there are different student types. Only a subset of students actually plan to complete a course, whereas the majority of students enrolled are there only to sample and explore. Measuring completion rate off the total number of students is misleading, but so is touting the total number of students enrolled (billions served) as a key metric. This is not a problem with Katy’s data, but with the common discussion of MOOCs in popular media and even higher education circles. I’ll address a different model in my next post.

Discuss here or at Google+ post.

Share Button

Google+ Comments

About Phil Hill

Phil is a consultant and industry analyst covering the educational technology market primarily for higher education. He has written for e-Literate since Aug 2011. For a more complete biography, view his profile page.
This entry was posted in Blogging, Guest Bloggers, Higher Education, Notable Posts, Openness and tagged , , , , , . Bookmark the permalink.

33 Responses to The Most Thorough Summary (to date) of MOOC Completion Rates

  1. It’s the backend of the donkey! http://cogdogblog.com/2013/02/25/back-end-of-the-donkey

    Percentages of finishers out of total entering open signups are the weakest, yet only (partial) measure we got. Completion needs to be tied to the intended goal of the learner is hopefully where you are going.

  2. Martin says:

    I love it when worlds come together – Katy is my phd student. Thanks for bigging up her work, she’ll be chuffed.

  3. Just because the MOOC providers don’t release this data doesn’t mean they don’t have it. They are commercial concerns and this is commercially sensitive data.

  4. Phil Hill says:

    Alan, yes, I plan to go more on different learner types within open education, each having different intended goals. By the way, nice post if disturbing image.

  5. Phil Hill says:

    Martin, after some research I discovered that Katy would be pleased:] some great work there.

  6. Phil Hill says:

    AJ, I’m sure the providers have this data, but I question the value of not releasing it. Open education is a different beast, and hoarding all data can be dangerous, especially if you make your living off of media narratives and momentum.

    At the very least, there should be some release from providers on how they are using such data to benefit learners. My $0.02.

  7. Pingback: De enorme aandacht voor MOOC's - Blog - Wilfred Rubens

  8. I am forced to wonder why we are focusing on “completion rate.” Do we know this construct aligns with “participant met his or her goals?”

    I am forced to wonder what the true “completion rate” of my undergraduate colleagues was those decades ago… did more than 10% (or so) complete? (I expect so) achieve his or her goals in every course? (I have my suspicions).

    Today, I reviewed a project completed by a participant in #edcmooc She posed to herself the question “Am I really learning in MOOC’s?” (my paraphrase). She answered her own question, “who cares.” She found her own utopia.

    Cipher with that :)

  9. Katy Jordan says:

    Hi Phil, thanks for featuring my work and helping to bring it to a wider audience. I’m really pleased to see all the discussions that it has sparked, both about openness of data, and meanings of success in the context of MOOCs :-)

    A lot of people have asked whether completion rates are the right way of framing the success of a MOOC; I agree that there is much more to the potential positive impacts of MOOCs for students than completion rate but, at the moment, completion rate is what the providers are measuring most consistently.

  10. Phil Hill says:

    Katy, thanks for your work. I agree with you that completion rate is the data available at the moment and therefore worth considering / analyzing.

  11. Pingback: The Four Student Archetypes Emerging in MOOCs |e-Literate

  12. Pingback: Emerging Student Patterns in MOOCs: A Graphical View |e-Literate

  13. ytabaa says:

    Thank you Katy for your great work
    Thanks to you too Phil for indexing it.

  14. Pingback: The Most Thorough Summary (to date) of MOOC Completion Rates | Computing Education Blog

  15. mgozaydin says:

    Thanks Kathy
    I just asked that yesterday and I found your message today .
    1.- Enrollment are high.
    Because
    1.1. Registration means just one click. I had done it 15 times so f ar .
    1.2. Students even do noıt bother to view the first lecture since they have no obligation. They did not invest any money, they will not get any credit. Just they will learn somehow something they wondered just for a moment during clicking for registration.
    1.3 Some, few, really wanted to learn something just go on .

    Then that is the result.
    From this application MOOCs are a failure : But only application is a failure .

    If a MOOC has credit and degree at the end
    People would be willing

    1.- To pay a reasonable amount , say $ 100 or so

    If they were going to be provided with a credit and a degree at the end .

    Only a 1,000 enrollment for 10 semesters at $ 100 would make any MOOCs
    effective. Then income in 10 semester is 1,000 x 10 semesters x $ 100 = $ 1,000,000 that is enough to cover any cost of that online .
    To continue MOOCs as it is waste of money and resources + waste of time of the sstudents too .
    Schools must
    Start charging something like $ 10 to $ 100 per course
    Start providing credits and degrees
    Satisfy themselves with 1,000 enrollment per semester and get better teacher students contacts
    Also makes money for themselves .

    Another model is
    Market MOOC courses among the 4,400 colleges in the USA at a small fee let

    the local college provide exams + credits and degrees
    Second model is easier to apply and a quick one too . Already there are some colleges doing that . Antioch + Colorado State Uni + San Jose State + some more .
    MOOCs are great if they are applied in the right way as I have explained above . It is wibn win case . Let us do not miss the boat .

  16. mgozaydin says:

    Let us be careful.
    There are good MOOCs and there are bad MOOCs .
    Investigate, ask your friends, then decide to get a MOOCs course .
    There are some very very good ones and also there are many bad ones .
    I usually go for the elite schools for security. Brand means good .

  17. mgozaydin says:

    Kathy
    Can you set up some organisation to get a complete data regularly covering all courses . I am willing to pay $ 100 per month subscription for such information . There may be many too . Thanks billion again .

  18. mgozaydin says:

    If Coursera will not decide on their business model to finance the initiative, I am afraid they will bankrupt soon .

    Edx non profit solved their problems .
    They will collect from their member providing schools $ 50,000 and or $ 10,000 plus 50 % of what they collect . If they help developing online course too then they ask for $ 235,000 per course . Not bad . Non profits
    became very profitable now . To me it is just not good . That should not be
    done to member schools .

  19. Pingback: The dirty little secret of online learning: Students are bored and dropping out – Quartz

  20. Pingback: the future of online learning looks a lot like the University of Phoenix

  21. Pingback: The dirty little secret of online learning: Students are bored and dropping out | SevenLearning

  22. Pingback: How would you measures success? | Learner Weblog

  23. Pingback: MOOCs: Progress Report

  24. Pingback: Wait a Second.. | Each One Teach One: Continuing the Conversation

  25. mgozaydin says:

    Katy
    Please what is the latest data . Thanks billion.

  26. Pingback: Learning Better | Each One Teach One: Continuing the Conversation

  27. Pingback: Lo que los MOOCs deberían aprender de la Comédie Française… y de Isasa Weis

  28. Pingback: Course Hero Adds 21 New Premium Courses! | Course Hero

  29. Pingback: Why Massive Open Online Courses Matter to Recruiters - ERE.net

  30. Pingback: What are MOOCs and why do they matter? |

  31. Pingback: MOOCs, Big Data, and the Open Web - Publishing @ SFU

  32. Pingback: The Patron Saint of MOOCs | pastpersistent

  33. Pingback: The Most Thorough Summary (to date) of MOOC Completion Rates – e-Literate | momeNTUm

Comments are closed.