Assessment in MOOCs

The more I read chapter in Macro-Level Learning through Massive Open Online Courses (MOOCs): Strategies and Predictions for the Future, the more I am starting to feel like Anton Ego from the animated movie Ratatouille ;-)  It's not that I am aiming to write harsh reviews of the stuff I read, but I kind of feel like the anticipation I have for reading some published things about MOOCs just aren't met with the appropriate level of satisfaction from reading what I am reading.

This time I am reviewing chapter 7, which is titled Beyond the Phenomenon: Assessment in Massive Open Online Courses (MOOCs).  The abstract is as follows:
MOOC course offerings and enrollments continue to show an upward spiral with an increasing focus on completion rates. The completion rates of below 10 percent in MOOCs pose a serious challenge in designing effective pedagogical techniques and evolving assessment criterion for such a large population of learners. With more institutions jumping on the bandwagon to offer MOOCs, is completion rate the sole criterion to measure performance and learning outcomes in a MOOC? Learner interaction is central to knowledge creation and a key component of measuring learning outcomes in a MOOC. What are the alternate assessment techniques to measure performance and learning outcomes in a MOOC? MOOCs provide tremendous opportunity to explore emerging technologies to achieve learning outcomes. This chapter looks beyond the popularity of MOOCs by focusing on the assessment trends and analyzing their sustainability in the context of the MOOC phenomenon. The chapter continues the discussion on ‘ePedagogy and interactive MOOCs' relating to ‘performance measurement issues.'

When I was a student in Applied Linguistics, for some of my courses I had professors who had us writing essays to practice answering questions in our field, but also staying on-point and not meandering, or including materials that were just not connected to the questions asked.  This was an invaluable exercise as it helped hone my skills as a writer.  Receiving peer review was also important, but I think having that in-class experience was really fundamental.   I think that it is this type of feedback that is missing in several chapters I've read thus far.

For example, this chapter (according to the abstract) asks what are alternative assessment techniques for MOOCs.  This is a good question! The author writes about the (on average) 10% completion rate in xMOOCs (the author is not that specific that they are xMOOCs, but from the text you can tell), and that other measures of what it means to complete a MOOC are necessary.  I completely agree with this position.  However, the author never really defines how completion is measured in those xMOOC contexts, and it's this that I find problematic. How can one start talking about alternatives (or additions to), when the current state is not really defined in terms of what assessment takes place in MOOCs to derive that magical completion number?  This is particularly important because the author  (in the solutions & recommendations section!) then goes on to describe CPR (Calibrated Peer Review) and AES (automatic essay scoring) which are used in coursera and edx MOOCs. These are tools used now to determine whether someone has (or has not) done what they need to do to be considered as completer. This doesn't really move the peg forward in terms of thinking about alternative (and alternatives to) assessment in MOOCs.

They talk (encyclopedically) about proctoring, MOOCs for credit, verified certificates, different MOOC 'types' (DOCC, BOOC, LOOC, SPOC, MOOR, SMOC), and digital badges (just to name a few things), but all of this is really disconnected from assessment in general.  Proctoring, credit, verified certs, and badges are by-products of assessment (not assessment types), and MOOC types don't really contribute much to the assessment discussion.  The language of MOOCs (i.e. the predominance of English) is discussed, but only really to suggest that existing assessment instruments (which only yield a 10% completion rate) be translated. OK. I don't disagree, but can't there be more substantive discussion here? How can this help more learners complete the MOOC. And, is a higher completion rate what we are looking for? Or is there more of a nuanced understanding of learning and assessment (and completion) in MOOCs?

I did chuckle a bit when I read that "a discussion forum is the main course component for active learner interactions and course participation in an online learning environment including a MOOC." (p. 125).  While xMOOCs tend to have forums, not all MOOCs are traditionally forum driven, and I would say that forums aren't the main course component for course activity.  I think by claiming this you are really framing MOOCs with the same frame as a certain type of online and distance education course for one thing.  It also predisposes one to think of activity (and what is assessable) in specific ways, ways which are defined by existing learning environments that have other underlying factors that influence and impact their design.

I really wanted to like this chapter (I really did :-) ), however between the disconnected information and the failure to deliver what what promised (or at least what I read into the abstract), it's hard to say that it's a must read.  That said.  If you completely ignore the title of the chapter and ignore the abstract, it's not a bad summary of current and potential topics to consider in the credentialing of MOOCs.

Have you read this? Your thoughts?



CITATION:
Chauhan, A. (2015). Beyond the Phenomenon: Assessment in Massive Open Online Courses (MOOCs). In E. McKay, & J. Lenarcic (Eds.) Macro-Level Learning through Massive Open Online Courses (MOOCs): Strategies and Predictions for the Future (pp. 119-140). Hershey, PA: Information Science Reference. doi:10.4018/978-1-4666-8324-2.ch007

Comments

Post a Comment

Popular posts from this blog

Latour: Third Source of Uncertainty - Objects have agency too!

Image
Continuing on my exploration of ANT, and asynchronous and indirect dialogue with Latour - this blog post will cover the third source of uncertainty, which according to Latour, is that Objects have agency too! As with the previous blog posts, I've pulled out quotes from the book that seemed interesting, or that I reacted to in some way, and I am responding to them here. no tie can be said to be durable and made of social stuff (p. 66) This quote seems to continue Latour's assertion that there is no such thing as "social" or "social stuff" and that "social", or the meaning of needs to be negotiated and better understood.  It also continues the thought that social can only be seen from the actions of its actors, the traces they leave behind, and that these bonds are not durable because they need continuous reinforcement. I guess Social is a perishable item. Left to its own devices, a power relationship that mobilizes nothing but social skil
Read more

MOOC participation - open door policy and analytics

The other day I was reading ZML Didaktik on the topic of MOOC participants. In MOOCs, one of the big questions is why are people lurking and not participating? If more than 500 people join a MOOC, why are only 10% contributing with any amount of regularity?  On the same blog, in a previous blog post, I had commented (it was an open stream of thought really) that perhaps there should be an open enrollment period, and then if the system sees that certain participants are bellow a threshold of activity, the system may give them the option to self-identify as a lurker, or un-register from the class. This line of thought went along a view that compared to the traditional classroom; in a traditional classroom students register for the course, they can attend classes for a week and then decide whether or not they want to stay with the course or not. If they do stay with the course they know that there is a certain amount of "lurking" that they can get away with, but they do have
Read more

You've been punk'd! However, that was an educational experience

Image
It's now the end of Teaching Goes Massive: New Skills Required (aka #massiveteaching) on coursera. Well, almost, we still have a couple of days left. I guess that the lesson here is that we were (the "learners") were punk'd† by Paul-Olivier Dehaye of the Univerisity of Zurich. After that last blog post (and subsequent pickup of the post by George Siemens and others) Inside Higher Education and the Chronicle wanted to chat with me about this MOOC experience. I only had time for one of the two before things went to press, so my 15 minutes of fame went to IHE.  Both IHE and the C hronicle have written about the topic, and have received some information from Coursera on the incident.  Others have also written about it (see here , here and here ) It's surprising to me that the University, and Coursera, waited until after this thing was a big issue in order to respond in the class, and clue people into what was happening. On Wednesday (day 3 of Week 3/3 of the
Read more