Tuesday, July 29, 2014

Perspectives on Late point deductions

I guess is teaching preparation time!  These past few weekends I've been going through my online course, updating due dates for assignments, and slowly starting to make the changed to the various modules that I had scribbled down as the course was in progress last spring.  It's still up in the air as to whether or not the class will run so I am thinking of applying for an assistantship for this fall semester.

In any case, in preparation for this course (if it runs) I've signed up for a variety of MOOCs on Coursera and on Canvas.net that deal with the subject of teaching online. I figure that this is a good opportunity for me to get some professional development, but also to discover any materials that I was unaware of. This way I can share these materials with my students (the course is about course design and teaching online).  My Pocket reader had filled with a lot of reading to go through and evaluate.  As I was reading some of the materials this one stood out to me: Enough with the late penalties.

This is actually something I've struggled with in my (brief) career as a professor (still sounds weird to call myself that). In the first few semester of teaching I didn't really deduct points for lateness.  I had a clause in my syllabus, but I never really exercised it unless the late submission was egregious, like being a few late days without an excuse.  Last spring I decided that to be fair and equitable I needed to really apply the late penalty on assignments equally across the board.  Thus, if an assignment was a day late, it got 5-points shaved off the top. Two days late? Another 5-points off, and so on.  What I noticed was this: The percentage of students who were late remained the same, they were just getting points off now.  The percentage of people who were on time remained the same.  There was only one small group that was late by 30-minutes every now and again as most adults have competing deadlines and sometimes fall behind on some things.

Going back to the article†, I agree in principle.  When we assess our learners we ought to be assessing what they are producing.  Are they demonstrating that they've learned what we set out for them to learn at the beginning of the semester? We are not assessing their timeliness in submitting their materials, so why take off points?  I suppose that I can add an objective to my course indicating timeliness, but are we working with adults here or not?  In the grown-up world there are deadlines with consequences. If you are late with the submission of an RFP bid your company many not get the contract. If you are late with your deliverables at work it may cost the company money, or in academia if you are late in contacting perspective students, chances are that they go elsewhere and you miss out on some potentially brilliant scholars in the making. There are consequences in life to being late, so why not apply this toward grades?

The article has a few suggestions, such as having a due date window where students can submit their work.  I am curious to know how this is different from a due date.  At the end of that window you have a firm deadline.  If you have advertised that an assignment is due on August 15th, and it is now July 10th, doesn't this give the learner enough time to plan? I wouldn't give someone more points if they submitted early, but what I would give is more feedback so that they can improve their submission and resubmit (for a better grade if they wished).

Having seen students who submit stuff late I am wondering what the best approach is helping the learner, but still getting stuff in on time because that matters.  I am not convinced that deducting points for lateness is not a good idea, but I am interested in the debate. 

If you teach, or if you are a learner, what are your thoughts?

† I would say that the article deals with children in a K-12 environment, but someone might make the argument in adult learning situations as well.

Monday, July 28, 2014

The perils of external rewards

A couple of years ago I was working on hashing out this idea of Academic Check-ins.  Think of it as Foursquare meets informal learning meets campus engagement meets alternative credentialing. A paper came out of that brainstorming with a proposal of what such a system might look like.  While working on hashing out some ideas I wanted to dive deeper into this concept of motivation, both internal and external.  One of the potential issues with extrinsic motivators such as the various "goodies" that you get for checking into places†.

While there wasn't a magic bullet (at least in the background research I did) for a good balance between internal and external motivators there was one huge warning: be very careful of external rewards for doing things. They slowly start to replace internal motivation that was there, and if you remove those external rewards, there is a danger of internal motivation not being there to sustain the learner.  This was somewhere in the back of my mind, but it really hit me yesterday as I was browsing the newly opened Learning to Teach Online course at Coursera. Now, I've been teaching online for the past three (or so) years, so I've gotten some of my own research done on this and I seek out communities of practice in this area to improve my own practice.  I decided to join this course as part of my own PLE.  When I logged in I saw that this course does not offer a Statement of Accomplishment (i.e. I was there, I read the materials and passed the quizzes), but they do offer a Verified Statement if I wish to pay for it.  This is the third Coursera course in two months that I have started that does not award a Statement of Accomplishment because (in my opinion) doing so would cannibalize their Verified Statement monetary opportunities.  While my final grade will be reflected on the course records page of my profile, I won't have a snazzy, but ultimately not very useful, statement to print out.

Now, this, for me anyway, has had an interesting effect on motivation, and how I approach these courses! This morning, during my commute to work, while reflecting on this, the aspect on motivation, extrinsic rewards, and what happens when you remove rewards that were previously given for a certain task, and the ultimate resulting detriment to motivation that this has on the learner. I have to say that now that Coursera has conditioned me to expect a Statement of Accomplishment for completing their courses I'm finding it hard to motivate myself when there is not statement of accomplishment available unless you pay for the Verified Statement.

Don't get me wrong, Udacity, FutureLearn and countless cMOOCs I've been part of in the past don't have statements of accomplishment, but I participated in them (in the forums too!) and really enjoyed the course. However when I entered those MOOCs I had no expectation of a statement of accomplishment.  My motivation was purely internal.  With Coursera (and any other instance where statements are issued) the motivation is internal when it comes to going through the material and browsing through the forums (passive participant in MRT parlance) but to get me to be active in that environment I need the carrot (statement or badge), especially when I am not a complete novice at the course. When that carrot doesn't existed and then it is removed, I am thinking much more judiciously about the amount of time I invest in any given course endeavor (i.e. not burning the midnight oil for graded assignments that won't give me something tangible to hold at the end).

I am simply reflecting on my own motivations here (so I don't expect a certificate), but I do wonder what adverse reactions might there be not only having paid certificates now that we've been conditioned to them. I've already caught a few stray comments on other courses indicating that they wouldn't have signed up if they know that there was no certificate for them.  Interesting.

Your thoughts on the matter?

† examples of this would be stickers in the case of GetGlue/TVTag, stickers in the new swarm check-in app, stamps in the old Gowalla, and some might argue badges would fit into this category too!

Tuesday, July 15, 2014

#MassiveTeaching experiment falls on deaf ears?

Alright, #MassiveTeaching (or under its official name: "Teaching Goes Massive: New Skills Need") on Coursera is over, that's all he wrote (and then deleted, and someone else recovered). All joking aside, I decided to participate in the final assignment/test of the course which ultimately turned out to be a Level 1 evaluation. I've included the three questions in my previous blog post about this. What I neglected to include in my previous blog post was a fourth question which went something like this: Would you recommend this course to someone else?

The requirement to "pass" the assessment was to grade 4 submissions. I ended up "Grading" 14 submission because I was really surprised at the results.

Of these 14 graded assignments, 8 of them were positive (57%) and these respondents claimed that they would, or have already recommended the course to their colleagues. One person commented that this course would be good for professional development and it was completed by them in mere hours. Another person got their previous professors "hooked" on this course (really?) The reasons for recommending the course are that learners felt that their peers would learn a lot about MOOCs, especially topics of pedagogy, copyright and technology. From a delivery point of view, others commented that they liked Paul-Olivier Dehaye's (few) videos (which were done in common craft style). One person would tentatively recommend the course to peers if Dehaye included more content for newbies.

Next up, of the 14 assignments I graded, 2 participants (14%) were unsure if they would recommend the course to peers. One response seemed to indicate that the whole drama around the course was a really a detractor, but there was a question about the efficacy of the course as an xMOOC. The respondent seemed to indicate that they got a lot out of the forum conversations and not Dehaye's work. The other response said that they liked MOOCs because they didn't have to structure their own learning, and the minimalism of this course was really not the format for them.

Finally, 4 of the 14 responses were negative (28%), in other words they would not recommend the course to their peers. The most common issue was the drama surrounding the course, and Dehaye's conduct. The interesting thing is that one of the respondent only learned materials through the videos, like going through some sort of OCW, which sharply contrasts with the comments another person made, in the unsure category, that stated that they only really learned something from the interactions.

Now this to me is interesting. Granted this is only a small cross-section of the responses but I was amazed at how positive the responses seemed to be overall. some people in the forums claim that they have graded over 40 to see what other said in their responses and they reported similar results. According to some sleuthing in the forums, there were 107 total responses to the final assignment. According to others the responses (yes/no recommendation) seem to be neck to neck depending on how you categories the responses. For instance do you could "yes only if..." as a "yes" or something else? What about responses like "your mileage may vary?"

From the 14 that I saw, it seemed to me that most people seemed to be new to MOOCs and as such had either no previous hands-on experience with MOOCs (only what they read in the press), or had minimal experience. This means, to some extent, that people's outlook on this #massiveteaching MOOC was predicated on three things:

  1. When you enrolled for the MOOC: If you enrolled past week 2 and missed the drama of content being deleted, you may have never noticed anything wrong (except for some angry people in the forums).
  2. Previous experience with MOOCs: Your previous experience might determine if you've been satisfied or not satisfied with this MOOC. Whether you are just watching the videos or whether you are attempting to participate in the forums.
  3. Paying attention: I got the sense that some people weren't paying attention to the MOOC. They seemed to indicate that they learned things about materials that were never released to the learners, like the Week 3 materials that never cam our way. I don't know how some people "learned" about the aspects of MOOCs that were never discussed neither in Dehaye's videos, nor in the forums. I wonder if these cases were cases of certificate hunters (like pokemon, gotta catch all MOOC certificates :) )


At the end of the day it seems to me that Dehaye's experiment fell on deaf ears. If people weren't perturbed enough to not recommend these rogue tactics I am not sure what to make of learners. Are they sheep willing to follow anyone in a position of authority? Are they not paying attention (mental image of student facebooking during lecture)? Are they trying to make lemonade out of a lemon? I really don't know. And for Dehaye, was all this production and drama all for naught? Your thoughts?


Thursday, July 10, 2014

You've been punk'd! However, that was an educational experience

It's now the end of Teaching Goes Massive: New Skills Required (aka #massiveteaching) on coursera. Well, almost, we still have a couple of days left. I guess that the lesson here is that we were (the "learners") were punk'd† by Paul-Olivier Dehaye of the Univerisity of Zurich.

After that last blog post (and subsequent pickup of the post by George Siemens and others) Inside Higher Education and the Chronicle wanted to chat with me about this MOOC experience. I only had time for one of the two before things went to press, so my 15 minutes of fame went to IHE.  Both IHE and the Chronicle have written about the topic, and have received some information from Coursera on the incident.  Others have also written about it (see here, here and here)

It's surprising to me that the University, and Coursera, waited until after this thing was a big issue in order to respond in the class, and clue people into what was happening. On Wednesday (day 3 of Week 3/3 of the course) the University posted the following:
Dear Coursera Students,
Prof. Dehaye, instructor of the Coursera course “Massiv Teaching - New Skills needed”, has deleted content during the course as part of his pedagogical concept in order to get more students actively engage in the course forum. In the course of the events confused students contacted Coursera directly, as they assumed a technical problem being the reason for the disappearing of course material.
Unfortunately, Prof. Dehaye had not previously informed Coursera of this part of his pedagocial approach: Deleting course material is not compatible with Coursera’s course concept, where students all over the globe decide when they want to watch a particular course video. Prof. Dehaye’s course included experimental teaching aspects which led to further confusion among students.
Coursera and the University of Zurich decided on Friday, July 3rd, to reinstall the course’s full content and paused editing privileges of the instructor until final clarification on the issue would be obtained.
The course is now back on track, and will conclude as planned, with the final assessment that is due this week. Once again, our apologies for the confusion and thank you for your patience.
- The University of Zurich Team

This response is quite interesting.  First it shows that Coursera has a "concept" of what MOOC teaching and learning looks like, and they are packaging it with their LMS. I guess the only sanctioned way to design and teach on Coursera is the Coursera way. This to me is quite problematic from a pedagogical stance; and I am sure others have written about this before and will continue to write about it. It's also problematic because it has shown that neither the University, nor the provider (Coursera) have some sort of way to address potential issues that crop up in these courses, especially the unexpected issues like the professor going rogue and deleting everything, or the professor just leaving the course and not telling anyone about it.

Now, we all knew that there was an experimental element to the class, Paul didn't hide it, but we weren't clued into the experiment. The experiment happened to us instead of having the learner play a controlling role in the experiment. If you know that you are supposed to set your own path, from the onset of the experiment, then you set your own path. If you think (given the Coursera model) that things function a certain way, when things don't, then you assume that they are broken and someone in a position of authority (i.e. someone with editing rights on the platform) needs to fix it so that we can all move on.

There have been a number of comments going around on research ethics of this whole situation. Personally I feel conflicted about this. I honestly don't think there was an ethical violation here on Paul's part. It doesn't stop me from feeling like this is a major facepalm moment, but I don't think he violated any ethics.  We all signed up willingly, and voluntarily (I hope!) for the Coursera platform, and then we again signed up for this "course." What makes me conflicted about this is the inevitable feeling of bait and switch that many people experienced. They were sold one thing, and they experienced something totally different.  It doesn't matter that this was a free course (a comment in the Chronicle article), so who the heck cares, the course was not free.  People spent their time and effort in an earnest attempt to engage with their fellow learners and what they got was "technical errors" (which were planned), confusion, and manipulation.

The result of this was really pedagogically ineffective. Paul's attempt really didn't address any of the stated learning objectives (saved on Scribd for those who aren't in the course), and it created a lot of ill will toward him, and the medium. There are better ways to help "disorient" educators to help them find their footing in the online landscape (the part in italics is one of the course "goals" - I guess it was written in invisible ink...). Anyone who has participated in a cMOOC can tell you that there are better ways. Each cMOOC is a unique, and initially provides for a disorienting experience until you get your footing, especially when you are new to MOOCs.  This is accomplished without pulling the rug under you, and by helping to create learner communities, learner initiated communities without the drama involved in #massiveteaching. This language of finding one's footing in the online world also seems to imply that people don't yet have a footing on the online landscape which in my opinion is a gross misunderstanding of the situation. What will probably happen is that participants in #massiveteaching will carry this experience forward and associate such shenanigans with Paul and possibly with online learning in general. Paul's tone-deafness to what the learners are saying, and the fact that he doesn't understand anything about existing literacies of learners (computer, critical, network, and so on) means that Paul has done a disservice to online learning everywhere.

In the IHE post Paul Dehaye is quoted as saying:
MOOCs can be used to enhance privacy, or really destroy it, I want to fight scientifically for the idea, yet teach, and I have signed contracts, which no one asks me about.... I am in a bind. Who do I tell about my project? My students? But this idea of the #FacebookExperiment is in itself dangerous, very dangerous. People react to it and express more emotions, which can be further mined.
OK, so I guess he was trying to make a statement about the dangers of the MOOC providers, and as Siemens points out they lack any governance structure and are leeching on the teaching functions of academic institutions. I'd love to say to Paul "Welcome to 2012."  Again, anyone involved with MOOCs, and is critical of the discourse around xMOOCs,  has known this for a while. Paul just showcased his ignorance, and further points us to the fact that this stunt was a bad way to go about raising awareness of the issues around xMOOCs. He lost more people than he gained. People will remember this stunt more than the stunt's message. It's a shame, because it is something we ought to be discussing.  Back in 2011 and 2012 I was really excited about MOOC research. I still am, but we only really see published research from cMOOCs.  Courses running on Coursera, EdX, and Udacity are like black holes. We rarely see any research coming out, and what does come out seems quite sanitized, like a press release.  What does happen to the massive data collected from student actions in a MOOC? Who controls this data? Who has access to it? Can we use it for pedagogically meaningful reasons?

Finally, in the course, I guess some people drank the cool-aid because this showed up in the discussion forums of the course:
In retrospect I think this MOOC was a brilliant example of Collaborative Problem Solving!! The challenge for me now is to think of problems that would engage my students as meaningfully as Paul managed to engage us. Real learning happened (whether we liked it or not), real emotions were felt and a huge bunch of strangers came together, offered our skills and helped the group work through the problem of learning online. Well done Paul and Coursera on opening my eyes to a whole new world of possibilities for learning!

Which was followed by a reply:
yes __NAME__, we already addressed the situation with CPS, if you looked at the FB site and the Google group of this course we took this course to a different level.. I am sure now all the members who took those have lost their motivation, This was infact my assignment asnwer in the Assesentment and teaching in 21st Centuray.. 
Part of me is wondering if this is falling in hook-line-and-sinker, or if this is making lemons out of lemonade, or if it's a coping mechanism on the part of some learners, a sense of getting something out of a giant disappointment. I think that there is something fundamentally wrong about claiming that you got something out of the course by going to an off-course resource to get something out of that course.  Now don't get me wrong, I've been in a lot of learning situations, in MOOCs, where a lot of learning occured on Social Media Sites like Facebook (Rhizo14 is a good example). However it was the learner's choice to go there (as was the learner's choice to go to the G+ group for Rhizo14 in lieu or in addition to Facebook).  It wasn't a forced path like was this case.   The learning communities of cMOOCs are created by learners, and the existence of such communities is aggregated through the network so that others may join if they wish. Learning on SMS is a grassroots behavior in cMOOCs.  In this case it seemed like the pre-determined path that Paul had set out, but only if the labrats (learners) went in the "appropriate" direction. While ethically it might be questionable, I find it pedagogically dishonest.

To wrap things up, the final "assignment" has been released, and it seemes a bit disingenuous to me, given everything that transpired.  It was a peer graded essay with the following three questions:
  1. Please state briefly what you knew or thought about MOOCs when entering the course. It might help to give some limited information about yourself (are you a student? an instructor? in what context?). This question is asked so your peers can assess your progress in the course.
  2. Please state briefly what you hoped to learn in this course. Again, this question is asked so your peers can assess your progress in the course.
  3. Did the student provide information about what they expected to learn in this course?

With regard to this "Assignment" I feel rather cynical on all fronts. On the one hand it feels like this is just another data-gathering stunt. So Paul ran his "experiment" and now he is collecting data to see what the learners say. The learners that didn't un-enroll from the course that is.  On the other hand, even if this is an earnest attempt to have learners introspect on this whole process, the attempt falls really flat on its face because this data is tainted.  The questions don't address anything that happened in the course. It feels like these were written with the original learning objectives in mind, and as such it reduces this final exercise into a farce. It is a farce that does not respect the learners, and it is a farce of the educational process.

Who is to blame for this farce? I don't think that the blame lies solely on one entity (and calls for the Paul's head are a bit over the top in my opinion). Coursera, University of Zurich and the Professor are equally to blame, as are the circumstances (i.e. lack of formal procedures). This was ill-conceived, there was apparently no appropriate or substantive communication between these three parties, no apparent oversight by some sort of curriculum committee at the University of Zurich, little-to-no oversight from Coursera (but they would love to charge you premium for verified certificates). At the end of the day, Paul, in my book, at best is a misguided academic, and at worst he is a troll. I think we have a long way ahead of us to make sure that such abuses of learner trust are not repeated again.

† Punk'd reference explanation for those who aren't aware of this show: wikipedia link.

Saturday, July 5, 2014

Social Experiment? Learning Experience? Tempest in a Teapot? Coursera's recently under-reported soap-opera.

Well, I am not quite sure what to make of this just yet, but I am keeping an eye on the situation to see how it gets resolved.  What situation am I talking about?  The seemingly under-reported (or not reported at all) situation happening in the course Teaching Goes Massive: New Skills Required, which is offered by Paul-Olivier Dehaye of the University of Zurich.  I have to say that initially the course description did not draw me in because anyone who claims that they will teach you about MOOC teaching is either naive, or selling snake-oil since the developments are so new.  I would prefer an approach, like a collaborative exploration (pioneered by a colleague), or something like #rhizo14 on the topic.  In any case, I re-read the description (below) and decided that I had 3 weeks to devote to the course.  My approach would have been a cMOOC style approach since the instructor didn't have a set syllabus. This way I could, potentially, continue to explore what it means to learn, teach, and design for MOOCs. The course description is as follows:
The course grew out of the author's experiences as an early adopter and advocate of newer technologies (such as Coursera) for online teaching. During the past year, the author often came in contact with higher education professionals who felt lost in the new landscape, and unprepared technology-wise. This is an attempt to educate anyone in this situation, and thereby empower everyone to anticipate changes in the field and make better professional decisions. Part of the emphasis will be to explain the technology, because it was and will be a catalyst of further evolutions in education.
Week 1  went well enough. From the interaction it seemed like this was a cMOOC/xMOOC hybrid of some sort.  There were some videos, in CommonCraft style, as well some external resources. The forums seemed to go well, so I decided to roll up my sleeves and participate a bit.  Then week two came, and the content was gone. No forums, no videos, no external resources, no people.  What the heck? A technical glitch?  No word from coursera or the instructor, neither by email, nor in the forums.  Well, coursera staff have said that they are looking into it, but nothing other than the boilerplate "we're doing what we can" response.

 The course "homepage" (for those unenrolled in the course) indicates that the course is no longer in session (although I am still enrolled) and that there aren't any upcoming sessions (see screenshot):

The professor is indicating, on twitter, that he has been removed from the course (I guess by coursera?), but there are no other facts around the interwebs:

Week 2 now is back, with some videos that Paul-Olivier produced. One of the compares coursera with edx, creative commons, copyright potential issues. I don't know if this is a no-no as far as the coursera terms of service go, but it would be interesting to go through them and have a look (whoever has time to read legalize of these MOOC LMSs).  Old threads are locked, which is a bummer because there was some nice discussion going on in some threads, and Paul-Olivier came back using a student account really briefly and then disappeared again.

The thing that I find most interesting is that neither InsideHigherEd.com nor the Chronicle have reported on this.  Back when GeorgiaTech's MOOC failed to launch, there was a lot of news coverage.  Now, perhaps the media, back then, wanted to discredit MOOCs (or pretend they are Nelson Muntz and say "ha-ha!" toward one effort that didn't go so well). Perhaps the media doesn't particularly care about MOOCs these days, but I certainly think that it was worth a brief mention somewhere.

Anyway, unlike GeorgiaTech's #FOEMOOC that shut its door to enrolled students as well (and left them without info), this MOOC, #MassiveTeaching, appears to have left currently enrolled students in the course, thus providing a fertile ground for some FUD to develop. Since the course was and experiment, this has given some people the excuse to claim fishing attempts in the course, or comparing the course to a massive experiment in the same group as facebook's recently controversial experiment. Some are also proposing that the course troll(s) may actually be Paul-Olivier running and experiment on them.  In any case, not much is known, and everyone is mum.

From a research point of view this is FASCINATING.  I would love to get a hold of the discussion forum data for both discourse and corpus linguistics analyses. On the other hand, I fear that coursera, and all involved parties, are handling this one wrong again.  We are now entering the third and final week of this MOOC on MOOCs.  Let's see how this pans out.

What do you all think about this?