Is it a course? Is it a textbook? No, it's a MOOC!

  by Flicktone

It's hard enough trying to evaluate regular education because learning is not easily measured; if indeed we can even hope to do so. We can of course test to see what people have remembered or whether they can explain concepts in a clear and logical manner but how do we know what they have actually learnt? So if evaluating traditional education is tough how hard is it to evaluate a MOOC? A bit like nailing jelly to the wall as a colleague once said, though I can't remember when.

MOOC providers may believe that they are offering a university course and duly organise it as such but do the participants share that belief? If you sign up for a regular university course you make a commitment, invest money and time and expect a reward for your efforts. You buy into the course concept, accept the university's terms and step into line. Everyone understands what they're involved in and accepts the rules. When you sign up for a MOOC, however, you don't buy into the concept in the same way. You join the course for your own reasons, have your own objectives and preferences and are often uninterested in the certification that may or may not be available at the end. Many have no intention of completing the "course" at all, they're simply curious to see what the fuss is all about. When there is no consensus as to what the MOOC is, an evaluation becomes extremely difficult.

Stephen Downes' post, Evaluating a MOOC, claims that we can't evaluate MOOCs with the same criteria as traditional courses since they aren't actually courses. They are many things to many people.

"I think the best way to understand success in a MOOC is by analogy with, say a book, or a game, or a trip to the city.
The person taking the MOOC is like a person reading a book, playing a game, or taking a trip to the city. It is impossible to talk about 'the objective' of such an activity - some people want to learn something (and others something else), others are doing it for leisure (and others as part of their job), others to make friends (and others to get away from their friends for a while), etc."

A successful MOOC creates enough momentum to continue after the "course" is complete. Since the motivations of each participant are so different it is more relevant to evaluate the MOOC as an entity and look at what the community has achieved and how vibrant and self-sufficient that community has become. MOOCs will develop into new forms and new terminology will emerge to differentiate between them. Some will be traditional courses with a syllabus and learning outcomes and will expect students to accept the rules of engagement whereas others will be more fluid and become learning communities with learners coming and going, some staying for months and others simply dropping in for inspiration.

"MOOC success, in other words, is not individual success. We each have our own motivations for participating in a MOOC, and our own rewards, which may be more or less satisfied. But MOOC success emerges as a consequence of individual experiences. It is not a combination or a sum of those experiences - taking a poll won't tell us about them - but rather a result of how those experiences combined or meshed together."

The important point is that evaluation is only possible when we know what it is we are evaluating and that all participants are aware of what they are engaged in. Confused? I still am but I'm trying to work it all out.