Tuesday, July 15, 2014

#MassiveTeaching experiment falls on deaf ears?

Alright, #MassiveTeaching (or under its official name: "Teaching Goes Massive: New Skills Need") on Coursera is over, that's all he wrote (and then deleted, and someone else recovered). All joking aside, I decided to participate in the final assignment/test of the course which ultimately turned out to be a Level 1 evaluation. I've included the three questions in my previous blog post about this. What I neglected to include in my previous blog post was a fourth question which went something like this: Would you recommend this course to someone else?

The requirement to "pass" the assessment was to grade 4 submissions. I ended up "Grading" 14 submission because I was really surprised at the results.

Of these 14 graded assignments, 8 of them were positive (57%) and these respondents claimed that they would, or have already recommended the course to their colleagues. One person commented that this course would be good for professional development and it was completed by them in mere hours. Another person got their previous professors "hooked" on this course (really?) The reasons for recommending the course are that learners felt that their peers would learn a lot about MOOCs, especially topics of pedagogy, copyright and technology. From a delivery point of view, others commented that they liked Paul-Olivier Dehaye's (few) videos (which were done in common craft style). One person would tentatively recommend the course to peers if Dehaye included more content for newbies.

Next up, of the 14 assignments I graded, 2 participants (14%) were unsure if they would recommend the course to peers. One response seemed to indicate that the whole drama around the course was a really a detractor, but there was a question about the efficacy of the course as an xMOOC. The respondent seemed to indicate that they got a lot out of the forum conversations and not Dehaye's work. The other response said that they liked MOOCs because they didn't have to structure their own learning, and the minimalism of this course was really not the format for them.

Finally, 4 of the 14 responses were negative (28%), in other words they would not recommend the course to their peers. The most common issue was the drama surrounding the course, and Dehaye's conduct. The interesting thing is that one of the respondent only learned materials through the videos, like going through some sort of OCW, which sharply contrasts with the comments another person made, in the unsure category, that stated that they only really learned something from the interactions.

Now this to me is interesting. Granted this is only a small cross-section of the responses but I was amazed at how positive the responses seemed to be overall. some people in the forums claim that they have graded over 40 to see what other said in their responses and they reported similar results. According to some sleuthing in the forums, there were 107 total responses to the final assignment. According to others the responses (yes/no recommendation) seem to be neck to neck depending on how you categories the responses. For instance do you could "yes only if..." as a "yes" or something else? What about responses like "your mileage may vary?"

From the 14 that I saw, it seemed to me that most people seemed to be new to MOOCs and as such had either no previous hands-on experience with MOOCs (only what they read in the press), or had minimal experience. This means, to some extent, that people's outlook on this #massiveteaching MOOC was predicated on three things:

  1. When you enrolled for the MOOC: If you enrolled past week 2 and missed the drama of content being deleted, you may have never noticed anything wrong (except for some angry people in the forums).
  2. Previous experience with MOOCs: Your previous experience might determine if you've been satisfied or not satisfied with this MOOC. Whether you are just watching the videos or whether you are attempting to participate in the forums.
  3. Paying attention: I got the sense that some people weren't paying attention to the MOOC. They seemed to indicate that they learned things about materials that were never released to the learners, like the Week 3 materials that never cam our way. I don't know how some people "learned" about the aspects of MOOCs that were never discussed neither in Dehaye's videos, nor in the forums. I wonder if these cases were cases of certificate hunters (like pokemon, gotta catch all MOOC certificates :) )


At the end of the day it seems to me that Dehaye's experiment fell on deaf ears. If people weren't perturbed enough to not recommend these rogue tactics I am not sure what to make of learners. Are they sheep willing to follow anyone in a position of authority? Are they not paying attention (mental image of student facebooking during lecture)? Are they trying to make lemonade out of a lemon? I really don't know. And for Dehaye, was all this production and drama all for naught? Your thoughts?


Thursday, July 10, 2014

You've been punk'd! However, that was an educational experience

It's now the end of Teaching Goes Massive: New Skills Required (aka #massiveteaching) on coursera. Well, almost, we still have a couple of days left. I guess that the lesson here is that we were (the "learners") were punk'd† by Paul-Olivier Dehaye of the Univerisity of Zurich.

After that last blog post (and subsequent pickup of the post by George Siemens and others) Inside Higher Education and the Chronicle wanted to chat with me about this MOOC experience. I only had time for one of the two before things went to press, so my 15 minutes of fame went to IHE.  Both IHE and the Chronicle have written about the topic, and have received some information from Coursera on the incident.  Others have also written about it (see here, here and here)

It's surprising to me that the University, and Coursera, waited until after this thing was a big issue in order to respond in the class, and clue people into what was happening. On Wednesday (day 3 of Week 3/3 of the course) the University posted the following:
Dear Coursera Students,
Prof. Dehaye, instructor of the Coursera course “Massiv Teaching - New Skills needed”, has deleted content during the course as part of his pedagogical concept in order to get more students actively engage in the course forum. In the course of the events confused students contacted Coursera directly, as they assumed a technical problem being the reason for the disappearing of course material.
Unfortunately, Prof. Dehaye had not previously informed Coursera of this part of his pedagocial approach: Deleting course material is not compatible with Coursera’s course concept, where students all over the globe decide when they want to watch a particular course video. Prof. Dehaye’s course included experimental teaching aspects which led to further confusion among students.
Coursera and the University of Zurich decided on Friday, July 3rd, to reinstall the course’s full content and paused editing privileges of the instructor until final clarification on the issue would be obtained.
The course is now back on track, and will conclude as planned, with the final assessment that is due this week. Once again, our apologies for the confusion and thank you for your patience.
- The University of Zurich Team

This response is quite interesting.  First it shows that Coursera has a "concept" of what MOOC teaching and learning looks like, and they are packaging it with their LMS. I guess the only sanctioned way to design and teach on Coursera is the Coursera way. This to me is quite problematic from a pedagogical stance; and I am sure others have written about this before and will continue to write about it. It's also problematic because it has shown that neither the University, nor the provider (Coursera) have some sort of way to address potential issues that crop up in these courses, especially the unexpected issues like the professor going rogue and deleting everything, or the professor just leaving the course and not telling anyone about it.

Now, we all knew that there was an experimental element to the class, Paul didn't hide it, but we weren't clued into the experiment. The experiment happened to us instead of having the learner play a controlling role in the experiment. If you know that you are supposed to set your own path, from the onset of the experiment, then you set your own path. If you think (given the Coursera model) that things function a certain way, when things don't, then you assume that they are broken and someone in a position of authority (i.e. someone with editing rights on the platform) needs to fix it so that we can all move on.

There have been a number of comments going around on research ethics of this whole situation. Personally I feel conflicted about this. I honestly don't think there was an ethical violation here on Paul's part. It doesn't stop me from feeling like this is a major facepalm moment, but I don't think he violated any ethics.  We all signed up willingly, and voluntarily (I hope!) for the Coursera platform, and then we again signed up for this "course." What makes me conflicted about this is the inevitable feeling of bait and switch that many people experienced. They were sold one thing, and they experienced something totally different.  It doesn't matter that this was a free course (a comment in the Chronicle article), so who the heck cares, the course was not free.  People spent their time and effort in an earnest attempt to engage with their fellow learners and what they got was "technical errors" (which were planned), confusion, and manipulation.

The result of this was really pedagogically ineffective. Paul's attempt really didn't address any of the stated learning objectives (saved on Scribd for those who aren't in the course), and it created a lot of ill will toward him, and the medium. There are better ways to help "disorient" educators to help them find their footing in the online landscape (the part in italics is one of the course "goals" - I guess it was written in invisible ink...). Anyone who has participated in a cMOOC can tell you that there are better ways. Each cMOOC is a unique, and initially provides for a disorienting experience until you get your footing, especially when you are new to MOOCs.  This is accomplished without pulling the rug under you, and by helping to create learner communities, learner initiated communities without the drama involved in #massiveteaching. This language of finding one's footing in the online world also seems to imply that people don't yet have a footing on the online landscape which in my opinion is a gross misunderstanding of the situation. What will probably happen is that participants in #massiveteaching will carry this experience forward and associate such shenanigans with Paul and possibly with online learning in general. Paul's tone-deafness to what the learners are saying, and the fact that he doesn't understand anything about existing literacies of learners (computer, critical, network, and so on) means that Paul has done a disservice to online learning everywhere.

In the IHE post Paul Dehaye is quoted as saying:
MOOCs can be used to enhance privacy, or really destroy it, I want to fight scientifically for the idea, yet teach, and I have signed contracts, which no one asks me about.... I am in a bind. Who do I tell about my project? My students? But this idea of the #FacebookExperiment is in itself dangerous, very dangerous. People react to it and express more emotions, which can be further mined.
OK, so I guess he was trying to make a statement about the dangers of the MOOC providers, and as Siemens points out they lack any governance structure and are leeching on the teaching functions of academic institutions. I'd love to say to Paul "Welcome to 2012."  Again, anyone involved with MOOCs, and is critical of the discourse around xMOOCs,  has known this for a while. Paul just showcased his ignorance, and further points us to the fact that this stunt was a bad way to go about raising awareness of the issues around xMOOCs. He lost more people than he gained. People will remember this stunt more than the stunt's message. It's a shame, because it is something we ought to be discussing.  Back in 2011 and 2012 I was really excited about MOOC research. I still am, but we only really see published research from cMOOCs.  Courses running on Coursera, EdX, and Udacity are like black holes. We rarely see any research coming out, and what does come out seems quite sanitized, like a press release.  What does happen to the massive data collected from student actions in a MOOC? Who controls this data? Who has access to it? Can we use it for pedagogically meaningful reasons?

Finally, in the course, I guess some people drank the cool-aid because this showed up in the discussion forums of the course:
In retrospect I think this MOOC was a brilliant example of Collaborative Problem Solving!! The challenge for me now is to think of problems that would engage my students as meaningfully as Paul managed to engage us. Real learning happened (whether we liked it or not), real emotions were felt and a huge bunch of strangers came together, offered our skills and helped the group work through the problem of learning online. Well done Paul and Coursera on opening my eyes to a whole new world of possibilities for learning!

Which was followed by a reply:
yes __NAME__, we already addressed the situation with CPS, if you looked at the FB site and the Google group of this course we took this course to a different level.. I am sure now all the members who took those have lost their motivation, This was infact my assignment asnwer in the Assesentment and teaching in 21st Centuray.. 
Part of me is wondering if this is falling in hook-line-and-sinker, or if this is making lemons out of lemonade, or if it's a coping mechanism on the part of some learners, a sense of getting something out of a giant disappointment. I think that there is something fundamentally wrong about claiming that you got something out of the course by going to an off-course resource to get something out of that course.  Now don't get me wrong, I've been in a lot of learning situations, in MOOCs, where a lot of learning occured on Social Media Sites like Facebook (Rhizo14 is a good example). However it was the learner's choice to go there (as was the learner's choice to go to the G+ group for Rhizo14 in lieu or in addition to Facebook).  It wasn't a forced path like was this case.   The learning communities of cMOOCs are created by learners, and the existence of such communities is aggregated through the network so that others may join if they wish. Learning on SMS is a grassroots behavior in cMOOCs.  In this case it seemed like the pre-determined path that Paul had set out, but only if the labrats (learners) went in the "appropriate" direction. While ethically it might be questionable, I find it pedagogically dishonest.

To wrap things up, the final "assignment" has been released, and it seemes a bit disingenuous to me, given everything that transpired.  It was a peer graded essay with the following three questions:
  1. Please state briefly what you knew or thought about MOOCs when entering the course. It might help to give some limited information about yourself (are you a student? an instructor? in what context?). This question is asked so your peers can assess your progress in the course.
  2. Please state briefly what you hoped to learn in this course. Again, this question is asked so your peers can assess your progress in the course.
  3. Did the student provide information about what they expected to learn in this course?

With regard to this "Assignment" I feel rather cynical on all fronts. On the one hand it feels like this is just another data-gathering stunt. So Paul ran his "experiment" and now he is collecting data to see what the learners say. The learners that didn't un-enroll from the course that is.  On the other hand, even if this is an earnest attempt to have learners introspect on this whole process, the attempt falls really flat on its face because this data is tainted.  The questions don't address anything that happened in the course. It feels like these were written with the original learning objectives in mind, and as such it reduces this final exercise into a farce. It is a farce that does not respect the learners, and it is a farce of the educational process.

Who is to blame for this farce? I don't think that the blame lies solely on one entity (and calls for the Paul's head are a bit over the top in my opinion). Coursera, University of Zurich and the Professor are equally to blame, as are the circumstances (i.e. lack of formal procedures). This was ill-conceived, there was apparently no appropriate or substantive communication between these three parties, no apparent oversight by some sort of curriculum committee at the University of Zurich, little-to-no oversight from Coursera (but they would love to charge you premium for verified certificates). At the end of the day, Paul, in my book, at best is a misguided academic, and at worst he is a troll. I think we have a long way ahead of us to make sure that such abuses of learner trust are not repeated again.

† Punk'd reference explanation for those who aren't aware of this show: wikipedia link.

Saturday, July 5, 2014

Social Experiment? Learning Experience? Tempest in a Teapot? Coursera's recently under-reported soap-opera.

Well, I am not quite sure what to make of this just yet, but I am keeping an eye on the situation to see how it gets resolved.  What situation am I talking about?  The seemingly under-reported (or not reported at all) situation happening in the course Teaching Goes Massive: New Skills Required, which is offered by Paul-Olivier Dehaye of the University of Zurich.  I have to say that initially the course description did not draw me in because anyone who claims that they will teach you about MOOC teaching is either naive, or selling snake-oil since the developments are so new.  I would prefer an approach, like a collaborative exploration (pioneered by a colleague), or something like #rhizo14 on the topic.  In any case, I re-read the description (below) and decided that I had 3 weeks to devote to the course.  My approach would have been a cMOOC style approach since the instructor didn't have a set syllabus. This way I could, potentially, continue to explore what it means to learn, teach, and design for MOOCs. The course description is as follows:
The course grew out of the author's experiences as an early adopter and advocate of newer technologies (such as Coursera) for online teaching. During the past year, the author often came in contact with higher education professionals who felt lost in the new landscape, and unprepared technology-wise. This is an attempt to educate anyone in this situation, and thereby empower everyone to anticipate changes in the field and make better professional decisions. Part of the emphasis will be to explain the technology, because it was and will be a catalyst of further evolutions in education.
Week 1  went well enough. From the interaction it seemed like this was a cMOOC/xMOOC hybrid of some sort.  There were some videos, in CommonCraft style, as well some external resources. The forums seemed to go well, so I decided to roll up my sleeves and participate a bit.  Then week two came, and the content was gone. No forums, no videos, no external resources, no people.  What the heck? A technical glitch?  No word from coursera or the instructor, neither by email, nor in the forums.  Well, coursera staff have said that they are looking into it, but nothing other than the boilerplate "we're doing what we can" response.

 The course "homepage" (for those unenrolled in the course) indicates that the course is no longer in session (although I am still enrolled) and that there aren't any upcoming sessions (see screenshot):

The professor is indicating, on twitter, that he has been removed from the course (I guess by coursera?), but there are no other facts around the interwebs:

Week 2 now is back, with some videos that Paul-Olivier produced. One of the compares coursera with edx, creative commons, copyright potential issues. I don't know if this is a no-no as far as the coursera terms of service go, but it would be interesting to go through them and have a look (whoever has time to read legalize of these MOOC LMSs).  Old threads are locked, which is a bummer because there was some nice discussion going on in some threads, and Paul-Olivier came back using a student account really briefly and then disappeared again.

The thing that I find most interesting is that neither InsideHigherEd.com nor the Chronicle have reported on this.  Back when GeorgiaTech's MOOC failed to launch, there was a lot of news coverage.  Now, perhaps the media, back then, wanted to discredit MOOCs (or pretend they are Nelson Muntz and say "ha-ha!" toward one effort that didn't go so well). Perhaps the media doesn't particularly care about MOOCs these days, but I certainly think that it was worth a brief mention somewhere.

Anyway, unlike GeorgiaTech's #FOEMOOC that shut its door to enrolled students as well (and left them without info), this MOOC, #MassiveTeaching, appears to have left currently enrolled students in the course, thus providing a fertile ground for some FUD to develop. Since the course was and experiment, this has given some people the excuse to claim fishing attempts in the course, or comparing the course to a massive experiment in the same group as facebook's recently controversial experiment. Some are also proposing that the course troll(s) may actually be Paul-Olivier running and experiment on them.  In any case, not much is known, and everyone is mum.

From a research point of view this is FASCINATING.  I would love to get a hold of the discussion forum data for both discourse and corpus linguistics analyses. On the other hand, I fear that coursera, and all involved parties, are handling this one wrong again.  We are now entering the third and final week of this MOOC on MOOCs.  Let's see how this pans out.

What do you all think about this?

Saturday, June 28, 2014

Tepid about Tenure

I am back home, and with vacation behind me I guess it's time to get back to work.  I've got the day-job for which I've already created a list of tasks to undertake; the teaching of my newly renumbered course INSDSG 684 (formerly 619) and the updates I want to make to the course materials as well as gearing up for my #altcred experiment, version 2.0; and finally the Great Big MOOC Book which I need to kick into high gear (two big things on the docket: additional reviewers and a publisher).

While I was away in Greece I participated in an online Orientation session for my EdD program which starts in August.  The timing was a bit brutal since I needed to be up at 03:00 (Greek time) to participate in an online session that was 18:00 MT. Regardless I did enjoy hearing from my classmates and professors, people who I will get to know more over the next four years.  One of the questions we were prompted to answer as part of our introduction was where do we see ourselves in five years after graduation.  This is one of my favorite questions, and I generally have an answer for it, but this time around I was a little perplexed and didn't have a concrete answer (well, one that I would personally consider concrete and actionable). 

Up until recently my answer for "what next?" was a tenure track job, probably in a College of Education, teaching in a program that focused on adult learning, online learning, and educational technology.  I still want to do this, but I am more tepid about tenure.  Tenure has been described to me as "job security" for those who have it, but it seems more like a velvet jail that you fight really hard to get into.  Just prior to leaving for vacation one of my colleagues told me that their "writing schedule" for this summer would be really hectic.  I guess it's "publish or perish" time because that's the only reason someone's unpaid summer months (remember faculty work September to May) would be crazy with writing.

Don't get me wrong, I like research. I like to explore questions that are of interest to me and see if I can find answers to them.  What I don't like is this artificial aspect of "you must publish x-many whatevers in y-rated venues in order to get to keep your job."  Academia is the only place I've seen people who have passed their probationary work period (usually a year in most professions), who have demonstrated good teaching and good work ethic for a number of years and still come up for additional scrutiny during their tenure review. Here your previously evaluated work (evaluated annually through a faculty review process in each department) gets evaluated again. What might have been passing marks in previous annual reviews may not land you in non-tenure (especially when department politics and interpersonal conflict are involved). This seems like an unnecessarily stressful position to put employees in.

And, once people are tenured, they can feel free to hang their hat and be done with any serious amount of work. I've heard stories of people who have just refused to take part in department faculty meetings (part of the "service to the department" category of your job duties), and there was nothing that the department could do about it because these individuals did not care about merit pay (something that comes out of annual review processes). Thus if you are content with your pay, you can stop all research activity, and all service to the institution and profession, and only focus on teaching two courses per semester (this is the regular course load at my University, I don't know what other schools are like).

The reason I am tepid about tenure isn't that I don't think I can make it, if ever I am in that situation, it's just that I think that the system is very very flawed. I would love to have a position in the future where I get to teach, where my working year is September to May (with a decent pay), and get to spend the summer doing whatever pleases me (research would be part of it, but it would also include a lot of rest). But, I don't think I can do it within a tenured context. I would like to have colleagues who respect the privilege of being a teacher, and respect their fellow colleagues, enough to be productive at work. Fighting for a tenured track job (the few that exist), and going through the gauntlet of "publish or perish" in order to keep your job, only to be in potentially dysfunctional department is a bit too much :-) In other professions you can pack up and go elsewhere, however in Academia when you get tenure (or if you are tenure track), if you decide to leave your current post you might be viewed as damaged goods (unless of course another school/department is trying to poach your from your current department/school).  In a currently tight labor market, that's not a good position to be in.

Anyone else in academia have thoughts about this? :-) Would love to hear them!

Monday, June 23, 2014

Trials and Tribulations of a book editor

Over the last few days I've been thinking about The Great Big MOOC Book, something that's been a project of interest since my first MOOCs (cMOOCs back then) and something I finally got the wheels off th ground, posted a call for chapters, even though I didn't have a publisher, got a number of great proposals that my two great colleagues, Rebecca Hogue and Alan Girelli, helped read, evaluate as well, and provide comments to the authors, and we're off! The call for papers also got the notice of JHU press, which was a nice compliment to my modest effort, and it seemed that we were nicely on track.

Of course plans change, things happen, and one of them being that I am starting a doctoral program at Atahabasca Univeristy this fall (EdD in distance education). I am wondering how much of my time The Great Big MOOC Book will take this fall semester. I've got 8/10 chapters in for review by me and my fellow reviewers. I was planning on having a rough draft ready for my trip to Athabasca this summer in order to give AU press a copy and see if they are interested in seeing if they would like to publish it (open access) but it looks like my new deadline for this should be around November or December. One of the reasons I want to pitch it to Athabasca is that they offer a free PDF, in addition to the paid paperback version. This was something that I had in the call for chapters as a feature of the book (it being open access).

In the meantime, I've had conversations with two people who've published books, through publishers, about their experiences. Since I am new to this, there is much to learn; both on the editing front, as well as the interface-with-the-publisher front. What I learned was that edited volumes, such as The Great Big MOOC Book, have a hard time with publishers because (at least from what I gathered from my two sources) academic publishers are in an academic monograph frame of mind when they review manuscripts, and edited volumes are much more difficult to get approved. I don't know if this is universally true, but it is giving me pause to ponder.

If finding a publisher (who wants to publish and offer an open access downloadable) of this book proves to be too much of a hassle, I seem to have a moral dilemma. On the one hand I may be able to get a publisher, but no open access. On the other hand, I have a potential venue for publishing that is open access, but it is an open access journal, not a book. It appears to me (at least from what I read in various places in academia) that people value book chapters more than journal articles (even if they are for a special issue). So the conundrum is this. Should I really spend a lot of time looking for a publisher that does OA, and risk the chapters going stale? Or should I pursue a more assured path to getting the research out through a special issue of a journal?

This experience has made me rethink the other book I was planning, the instructional design of MOOCs, for which I have already got 12 tentative chapter titles and topics. What are your thoughts?

The silver lining here is that this experience is giving me ideas and potential paths for future jobs and careers after the doctorate is done (since tenure track jobs are an endangered species).