San Jose State University (SJSU) and Udacity have announced the results of their summer pilot, and the headlines cover the big improvements (text from IHE article, table from Udacity blog).
Thrun recently hinted that the summer pilot’s results would be more positive, and that Udacity was getting close to finding the “magic formula” to deliver high-quality, low-cost education.
The lone holdout among the SJSU Plus courses is entry-level math, which saw the smallest increase in students who received a passing grade, from 23.8 to 29.8 percent. That places the pass rate almost 40 percentage points below the closest SJSU Plus course, and about 15 percentage points below the pass rate of the on-campus course.
I am not trying to throw cold water here, but it is very important to look at the student populations. To his credit, Sebastian Thrun describes the big differences in his blog post.
This summer, we ran the second instance of our pilot. While in the Spring, we actively sought out underserved high schools from low-income areas in California, this time we simply opened up enrollment to anyone. As predicted, with 2,091 students who enrolled, we mainly reached students who would not ordinarily attend college. Only 11% of the summer students who took the for-credit courses from SJSU were matriculated students in one of the California State Universities. 71% of our students came from out of state or foreign countries. And while the total number of high school students went up, their proportion in the total student body went down. [snip]
One key difference between Spring and Summer was that we opened our Summer session to everyone. This led to a substantial difference in student body. Among the student body, 53% reported that they already hold a post-secondary degree (5% Associate, 28% Bachelor’s, 16% Master’s, and 4% Doctorate). Only 12% of the students had a high school graduate diploma or equivalent, and 15% were active high school students. This is very different from the Spring Pilot, in which approximately 50% of the student body were active high school students, and the other 50% were matriculated SJSU students. [emphasis added]
One of the key success factors of online education is to target student populations who can succeed in this environment. If you target groups (remedial students, underserved students, high school students), then there must be tremendous student support.
Pay attention to the student populations – this might be the real story. If you ignore the difference in student populations, you might draw the wrong conclusions.