Week 7 - Evaluation (OLDSMOOC)

It's week 7 in OLDSMOOC, and as we are windowing down we are tackling the topic of Evaluation. I will be switching tracks again, from the Blended Mobile Learning course (that I've been working on for a while), and going back to the idea of offering the course as a cMOOC. Going through OLDSMOOC I've gotten some good ideas about how to implement my own cMOOC.  I've been thinking a lot about the recommended paths that are available in some weeks (the short and the long path). This, in combination with badging, and deliverable, is making me think about the assessment aspect of the MOOC; but let's not get sidetracked, let's talk about evaluation.

In terms of evaluation decisions, what immediately comes to mind are these:
  • Should the content for this learning design be expanded, reduced, or remain the same? One of the tricky things about MOOCs is that you will always have critics, since there will be instances (many of them) where the MOOC does not hit the sweet spot for quite a few people.  How does one address these issues, and how does one stay explain the scope of the MOOC to the stakeholders?
  • What is the best way to make the materials open to anyone who may want to facilitate this course? I am thinking about this course as a way to also contribute back to the community, thus I want to make the OER as accessible as possible.
  • What strategies can we identify to maintain our learning design in the future? If the MOOC is going to run more than once, then, like open source software, there needs to be some way of maintaining the learning design, and updating it
  • Should the MOOC offer assessment instruments?  If so, what are meaningful assessment instruments and, when implement, will they have met their goal?
  • What is our engagement goal? Learning does not happen in a vacuum, so how do we encourage people to engage with one another without being overbearing? You can't force engagement and cooperation onto others.
  • How did facilitators work out? How was communication between facilitators and participants?  What worked and what did not?
  • Does one facilitator work? Or should we employ many more facilitators?
  • What is the "sweet spot" for MOOCs? 4 weeks? 6 weeks? 8 weeks? 12 week? If students are taking this for credit (like they did with CCK) then their academic semester is defined, and they need to "slog through" it, but what about the open learners?
  • What role should learner supplied resources take in the course if the course is offered for credit? In MobiMOOC 2011 there were participants that posted resources like there was no tomorrow (this was a good thing in my opinion), but what do other participants feel about this?
 As I was writing this, I decided to mingle the specific question in with the decisions to be made.

Your thoughts?

Comments

Apostolos K. ("AK") said…
Apostolos, you have identified an interesting set of evaluation decisions and questions. As you noted, you have mixed decisions and questions together. This is OK, but I generally promote the idea of identifying decisions at one level and then having one or more questions
aligned with each decision to be made. Evaluation planning is difficult to be sure. When you are wearing your evaluator hat, you may find yourself in the position of having to persuade your collaborators of the value of
anticipating difficult decisions, asking certain questions, or addressing specific issues by engaging in evaluation. Unwilling or unable to confront the complexities involved in most evaluation contexts, some people may demand
direct and simple answers to complex questions. But direct and simple answers are extremely rare in the context of learning design, and “it depends” and
other conditional statements are part of even the best evaluations. A thorough evaluation plan will expose as many of these conditionals as possible up-front, but the trick is doing so without having the team decide to abandon evaluation altogether.

One of your decisions is: “Does one facilitator work? Or should we employ many more facilitators?” My
experience in online teaching is that you need at least two facilitators, and most MOOCs have several (as the OLDS MOOC does). The biggest issue is that one
person can have an adverse event (e.g., illness, death of a family member, hurricane, etc.) occur during the course, and be unable to keep up with the demands of the course. Here is a link to a MOOC focused on Learning Analytics and Knowledge that my friend, George Siemens, kicked off earlier this month: https://www.canvas.net/courses/learning-analytics-and-knowledge You’ll notice that he has six world class facilitators for eight week course.


Good luck with your project and your upcoming PhD studies.
February 24, 2013 at 4:12 PM
Post a Comment

Popular posts from this blog

Latour: Third Source of Uncertainty - Objects have agency too!

Image
Continuing on my exploration of ANT, and asynchronous and indirect dialogue with Latour - this blog post will cover the third source of uncertainty, which according to Latour, is that Objects have agency too! As with the previous blog posts, I've pulled out quotes from the book that seemed interesting, or that I reacted to in some way, and I am responding to them here. no tie can be said to be durable and made of social stuff (p. 66) This quote seems to continue Latour's assertion that there is no such thing as "social" or "social stuff" and that "social", or the meaning of needs to be negotiated and better understood.  It also continues the thought that social can only be seen from the actions of its actors, the traces they leave behind, and that these bonds are not durable because they need continuous reinforcement. I guess Social is a perishable item. Left to its own devices, a power relationship that mobilizes nothing but social skil
Read more

MOOC participation - open door policy and analytics

The other day I was reading ZML Didaktik on the topic of MOOC participants. In MOOCs, one of the big questions is why are people lurking and not participating? If more than 500 people join a MOOC, why are only 10% contributing with any amount of regularity?  On the same blog, in a previous blog post, I had commented (it was an open stream of thought really) that perhaps there should be an open enrollment period, and then if the system sees that certain participants are bellow a threshold of activity, the system may give them the option to self-identify as a lurker, or un-register from the class. This line of thought went along a view that compared to the traditional classroom; in a traditional classroom students register for the course, they can attend classes for a week and then decide whether or not they want to stay with the course or not. If they do stay with the course they know that there is a certain amount of "lurking" that they can get away with, but they do have
Read more

You've been punk'd! However, that was an educational experience

Image
It's now the end of Teaching Goes Massive: New Skills Required (aka #massiveteaching) on coursera. Well, almost, we still have a couple of days left. I guess that the lesson here is that we were (the "learners") were punk'd† by Paul-Olivier Dehaye of the Univerisity of Zurich. After that last blog post (and subsequent pickup of the post by George Siemens and others) Inside Higher Education and the Chronicle wanted to chat with me about this MOOC experience. I only had time for one of the two before things went to press, so my 15 minutes of fame went to IHE.  Both IHE and the C hronicle have written about the topic, and have received some information from Coursera on the incident.  Others have also written about it (see here , here and here ) It's surprising to me that the University, and Coursera, waited until after this thing was a big issue in order to respond in the class, and clue people into what was happening. On Wednesday (day 3 of Week 3/3 of the
Read more