Monday, August 31, 2015

MOOC Cheater! I caught you!

This past week the web was abuzz with new research to come out of Harvard and MIT on cheating identification in MOOCs, specifically xMOOCs hosted on the edX platform, but I suspect that any platform that collects appropriate analytics could see this used.  The title of the paper is Detecting and Preventing "Multiple-Account" Cheating in Massive Open Online Courses and it's an interesting read. I find the ways of crunching data collected by web-servers as a way of predicting human behavior fascinating.  While I am more of a qualitative researcher at heart, I do appreciate the ways in which we can use math, data, and analytics to derive patterns.

That said, my main argument with the authors of the article are not the methods they use, but rather the actual utility of such an algorithm.  The authors write that CAMEO (Copying Answers using Multiple Existences Online)† is a potential threat to MOOCs because

  1. CAMEO is highly accessible. Anyone can create additional accounts, harvest answers, and not be dependent on someone else to provide the cheater with the answers.
  2. Anyone can sit down in one sitting and acquire certification for a MOOC
  3. Therefore cheating via CAMEO can really lower the value of MOOC certificates, or just render them valueless.
  4. As an added bonus, CAMEO, it is pointed out, is counter to the ToS of the various xMOOC providers.
While I think that the process is interesting, I think that the authors' cautionary tales are part FUD and part bunk.  Yes, CAMEO is accessible to everyone.  If I had nothing better to do I would most likely create a few more accounts on Coursera and edx so I could ace my tests.  So what? It doesn't mean that I learned anything, and on top of that edx institutions have no (or little) skin in the game.  The reason why cheating is treated so seriously on campuses is because Universities lend their names and reputations to the students who graduate from their schools. Thusthe learners gain credibility by virtue of the credibility of the school.  I have not seen this happen in MOOCs yet.  MOOCs are not treated the same, at least as far as credibility goes, as traditional school environments.  I am not saying that they can't be, but they are not now.  In instances where we come closer to having the same level of skin in the game, we have verified certificates where people are proctored when they take these online exams.

The second issue, of being able to sit down in one sitting and get a certificate, is really a non issue. Some people already know the stuff that is covered in the MOOC, but they don't have a way to show that they already know the stuff.  Going through a MOOC where they can just sit down and take the assessments (if all of them are multiple choice anyway), means that in a relatively small time-span they can get some sort of acknowledgement of their previous knowledge.  There is nothing wrong with this.  This actually happened to me last summer.  I was interested in the Intro to Linux MOOC on edx.  Once the thing started I realized that through my peripheral linux use over the past 15 years I already knew the basics.  The course wasn't directed toward me, but I ended up taking the tests and the exams (which seemed easy) and I passed the course way before the closing date.  I suppose that the course rekindled the linux flame and got me using Ubuntu on a daily basis, but from just a course perspective I could be considered a cheater if concern #2 is one thing that pulls cheaters to the forefront.

Finally, the worry about diminishing he value of the certificate of completion...Well... hate to burst your bubble, but I would argue that certificates of completion for MOOCs are nice little acknowledgements for the learner that the work was done, but in real life they have little meaning to anyone but the learner. A certificate of completion may mean something to a current employer who may have asked you to undertake some sort of course, but it's really just a rubber stamp.  The rubber meets the road when you need to apply what you've learned, and neither a MOOC, not traditional corporate training (for that matter) can ensure that you can perform.  There need to be additional on-the-job support mechanisms available (if needed) to make this happen.  A certificate just means that you spent the requisite amount of time in front of a computer and you got some passing grade on some assessment (well, maybe - some corporate trainings have no assessments!).  At the end I wouldn't worry about the diminished value of a certificate of completion because it has no value.

To be fair, the authors do talk about the limitations of their findings, such as only having suspected cheaters, and not having confirmed their suspected cheaters with reality, but they also talk about the reality of trying to prevent "cheating" in MOOCs.

I would have found this paper much more interesting if it weren't so value-laden and steeped in preventing "cheating" in MOOCs.  Cheating, to me anyway, means that you have something substantive to gain by taking the shortcut.  In learning the only substantive thing to gain is the knowledge itself, and there is no shortcut for that (unless someone has invented a matrix-style knowledge dump machine and I can learn kung fu now).

Your thoughts?


NOTES:
† There is a line in the pilot episode of Marvel's Agents of S.H.I.E.L.D. when Agent Colson asks Skye if she knows whatStrategic Homeland Intervention, Enforcement and Logistics Division means and she responds that someone really wanted it to spell SHIELD.  I couldn't help but think about this when CAMEO was spelled out..
blog comments powered by Disqus