Friday, August 29, 2014

MOOC Completion rates matter?

A while back I came across a post by Martin Weller titled MOOC Completion rates DO matter. Because my Pocket account was overflowing with some great content (including this one), I thought it was high time that I read this article ;-).  In this short post Martin writes that completion rates do matter in MOOCs, taking the opposite view of some cMOOC folks. 

He goes on to tackle the analogy that MOOCs are like newspapers and that people don't necessarily read all sections, or even visit all sections.  As someone who doesn't really read physical newspapers that often any more (I only bought one last summer so I could get the Asterix comic that came with it), the analogy only gets me so far.

I don't know what sort of analogy is valid for MOOC participant behaviors.  After pondering this a bit, I don't think that there is one analogy that will encompass all behaviors.  At the beginning of the MOOC I think the analogy of window shopping is most appropriate for everything leading up to the conclusion of the first (or maybe even the second) week of the course.  I think that there are many people who are curious about the course and join it, regardless of their intents for the course.  Coursera and EdX have gotten a little smarter these past few months and query their users to see what their intents are for the course right at the beginning of the course.  Are they interested in just browsing the content? browsing the content and taking some quizzes? doing the assignments? going full force into the course?  I think this is smart, because at the onset of the course you have idea of what the participants in the course are intending to do (assuming that they are truthful in their answers).

This only goes so far, however.  Back in the days of Change11 I had pondered about having a nag-system (for lack of a better term) that would keep track of user logons in the system, and track their patterns of participation.  If they weren't active the system would send them a reminder email about the course, and see if they would like to unenroll, so there would be a perpetual Opt-In function for people who are not active. I think at this point I would also augment the system with highlights of upvoted posts and interesting discussions happening so as to entice people to come in and say something (or at least if they don't contribute something, they will be seen by the system as being active). Lurking isn't bad, everyone lurks from time to time, it's not possible to fully engaged all the time, in each course you take.  You do need time to let things steep.  Especially in MOOCs you might have multiple conflicting priorities and need to put the MOOC on the back burner for a few days. That said, the point of the system is to separate those who are passive participants (aka lurkers), from those who are checked out (and not coming back), and from those who are on leave (not active for a week, but intend on coming back).

Then, at the end, it would be useful to have participants self-assess.  Those who took the initial survey should be surveyed again to see if they met their own expectations, or if they did or did not meet their own levels of engagement in the course.  The other points to ponder are theses:
  • Under whose rubric do we measure completion? The course designer certainly has some ideas about what it means to complete a course, however in an open course you don't control for enrollment, so you might have some people who are very under-prepared for the course, and some where the course is too basic for them.
  • Does completion equate with learning?  These two aren't necessarily the same. Going back to the case where the participants already have the knowledge, but don't know it here's something from my own experience:  I was in two courses, one in coursera, and one in edx.  I signed up to learn something new, only to find out at the end of Week 1 that things seemed a bit basic.  I ended up taking the quizzes for these two courses in advance for subsequent weeks (they were available) and I passed with high 90s without viewing any subsequent lectures past week 1.  I completed the courses, but did I learn anything (new)?  The answer is no.
  • Who does completion matter to? I know institutions who subsidize these want to see some ROI, but ultimate, in an open course (where there is no formal grade or credit for that grade), doesn't the participant sit in the drive seat to determine whether they've completed the course or not (to their own satisfaction)?

Do completion rates matter?  I think they do, however in certain specific contexts.  Not all contexts are the same!  What do you think?




comic from: http://completelyseriouscomics.com/comics/2011-01-29-Pondering.jpg

Wednesday, August 27, 2014

Group presentations and meeting faculty

Slowly catching up and getting back to normal, although I suspect with the semester beginning next week at work we'll be on a different sort of normal for the next few weeks.  I thought it would be a good idea to continue my blogging debrief of my orientation experience for Athabasca's EdD program that I did last week in Edmonton.

Part of the orientation experience is presenting the first assignment while you're there.  The first assignment for EDDE 801 is a collaborative presentation.  In May we were informed of which team we would be in, and we essentially had from mid-May to mid-August to choose a topic and then negotiate as we saw fit what we would present in the 60 minutes that we were alloted.  Our team had a choice of Andragogy and Multimedia in education - we opted for Andragogy after a little discussion.  We were given some general ideas of what we were expected to present during this collaborative presentation, but it was open ended.  Part of the reason is that we would be coming back to these topics again in subsequent weeks in the course, so if we weren't completely exhaustive in the presentation it was alright. I think that the emphasis of the presentation is more on process than on content.  The content needs to be there, but the process of getting your materials ready, and getting your team ready, and then presenting, is a much more important part of this assignment. That said, I think we did an awesome job with the topic of Andragogy (of course it should be noted that I am biased ;-) )

The overall planning for the presentation was interesting. We had not met each other in person before, so working together asynchronously, and sometimes synchronously, to get this done reminded me a lot of the collaborations that I've had with the MobiMOOC research team.  I think that the experience of working with others in a distributed fashion to produce something new was really instrumental in preparing me for this first assignment.

The other part of the orientation was to really get to know the faculty of the program. Some were there with us, while a few were remote.  I have to say that the most confusing aspect in this set of presentations over the 3 days where we were learning more about the processes involved in being an EdD student was what to call people. Do you go with Pat, or Dr. Fahy? Terry or Dr. Anderson? (and so on). When in doubt, I guess the more formal is the default form of address, but when you've read their work over the years (and had your own graduate students read it), and you've interacted with them over the web (as is the case with George Siemens) you get a sense that you know them a bit more than one would know them in a typical orientation setting.  That said, while the presentations that they all had were interesting, we did have a handful of the faculty with us for longer than their presentations.  I think more in-depth conversations could were had when they weren't presenting than when they had their powerpoints up on the screen.

While I was talking to Marti (Dr. Cleveland-Innes) I started drifting toward the course I teach (INSDSG 684) and this process I use and model for the students...and I blanked.  What was the name of the people who proposed this process?  I could describe it perfectly fine, but the names escaped me.  Well, that was embarrassing.  Back at the hotel room I was thinking hard and it came to me Con...rad....and....Do...Dona...Donaldson (Phases of Engagement). Things come to you at very odd times.  The orientation was a bit of cognitive overload, so I am not beating myself up too much about this, but you don't want to look like an idiot in front of one of the people who wrote about the Community of Inquiry framework ;-)

Monday, August 25, 2014

Of Cohorts and Residential Requirements

Back in Boston!  I was off to Edmonton last week for my doctoral program orientation at Athabasca University. The orientation is a formal part of the first course (EDDE 801) and it is a requirement.  Not attending the orientation means not being the program.  Those who know me on campus know that I am not a fan of cohorts, and I don't like residential requirements, so it might seem illogical to apply to a program that requires both.  Well, what can I say? I thought I would try something new for a change and go outside of my comfort zone and see what it's like (who knows, maybe I'll re-assess my stance on cohorts and residential requirements).

The incoming cohort each year (this year we are cohort 7) is made up of 12 members.  We actually had 13 members to start off with, but one of us was wooed to go an join the DBA program at Athabasca, so he is gone from our cohort, but I consider him an honorary member. Apparently the optimal size of a cohort is 20-30 students (from one of the team presentations), so our cohort seems to be the minimum acceptable cohort size. So long as we all stick together we'll be fine.  I don't quite know how I feel about the cohort model yet, however  having only 11 more people in our cohort makes it easier to remember faces and names, and get to chat with others to a point where you get to know them and they can be your support network.  Most of my Master's level coursework had around 20 students, and while we started with several classes together (and had an opportunity to get to know one another), the people I remember the most are people I worked in teams with, so fewer people for me means more opportunities for mixing up the teams, and therefore getting to know people better.  Apparently, this year, Athabasca accepted one-third of the qualified candidates into the EdD program.  I guess we could have had an optimal cohort size, but smaller seems better to me at this point in time.

As far as residential requirements go, I am a bit torn about this one (at this point in time).  I am exhausted from the trip. Getting there (and finally resting) was a 24 hour day for me (luckily I had a day to recuperate). Getting back was an 18 hour day.  It's odd, but getting from Boston to Edmonton is not an easy feat! I ended up with 5+ hour layovers in Dallas (which wasn't a bad airport actually). From a travel perspective the residential requirement was a drag.  That said, while I didn't get to hang-out with everyone for the same amount of time, I think I did get a good sense of who everyone is, where they are coming from, and what their goals are.  I think that having those shared in-person experiences, both in the classroom, and outside, are quite important. Edmonton was also pretty nice, so we had a choice about which places to go to each afternoon once the classroom component was complete.  I have revisited my thoughts on residency: I think that a residency can be a good thing, but it needs to be planned and designed right.  A residency requirement that is completely flexible as to when you do it is almost useless because you don't have an opportunity to form bonds with your cohort early enough that it makes a difference in your studies.  I also think that what you do, in the residential component, is important.  If it can be done online (and better) why do it in person?

Things to ponder.

Friday, August 15, 2014

Should faculty be 12-month employees?

I guess today I will be taking off my "Instructional Designer" cap, and putting on my "Higher Education Administration" cap. My career in higher education goes back to the days of me being a  work-study student, working for the department of Media Services, providing all those nice A/V equipment that professors use as part of their course.  Since then I've had a variety of jobs with an ever increasing responsibility load.  Despite the change of departments, change in job descriptions and duties, I always remained outside of an academic department (you know, the ones that have professors and teach courses).  I was always in some support role, and usually one that involved technology.  I have always been a 12-month employee, not an academic-calendar employee (9-month, September to May) like the TT† faculty. If you had asked me back then if I would want to take the entire summer off, I'd probably tell you that you were nuts. Even if we factored in the lower pay, the summer is when business gets done!  You speak to vendors, you try out new products, you upgrade your current bits-n-bobs for new trinkets that will support the teaching and administrative function of the university.  In short, the summer is when you can experiment and not impact a lot of users as you are tweaking your services.  I guess the analog would be that we are the elves working tirelessly to prepare for Christmas.

Then I moved to an academic department, which I really love.  There is only one thing I was not prepared for:  I wasn't really sure what the impact would be of faculty (including the department chair) being 9-month employees.  Faculty are the life-blood of a department, they aren't just warm bodies who teach courses.  They are subject experts who bring their wealth of knowledge, curiosity, and energy to the department.  They are the driver (or supposed to be anyway) of new innovative offerings, of new partnerships, and of course, mentoring the next generation of (insert profession) scholars and practitioners. However, they aren't around in the summer!  We (the program administrators and secretarial staff) are around, and we get done what is within our sphere of influence, however we can't do all things alone. The faculty are part of our team, and they need to be involved in major decisions, such as partnering with other department, working on ties with other Universities, arranging for symposia and so on.  These are important things that can be done in the summer, but because faculty don't work in the summers (they aren't paid in the summer), this important work doesn't get done.

Now, granted, some of you may say that committee meetings and things like that get done in the academic year, but I would argue that September is hectic as the semester starts and it's probably too much work to throw to faculty.  December is the Holiday break (and for us capstone grading!), January no one's around, and May we're back to vacation and final exam grading modes.  This really leaves five months, out of the year, to be super productive.  Anyone who's worked in management knows that you can't just condense a year's worth of work into 5 months.  Partnerships take time to build, paperwork and legal documents (if needed) also need time, Deans and Provosts need to approve some things which means that they also need their time to consult and go over things.

So, my (potentially naive) question is: should faculty be 12-month employees.  Sure, they can choose to take vacation like the rest of us, but should they be on the hook for the summer months to do committee work, prepare proposals and documentation for program offerings, program improvement, and spend the summer (when they don't teach) some quality time with the non-faculty staff planning out the next moves that will make their programs competitive and reinvigorated?

Your thoughts?


† TT = tenured, or tenure track.

Wednesday, August 13, 2014

Educational Based Research - Part 1


Well, in a week I will be in Edmonton starting off my EdD in distance education at Athabasca University.  I know that most North American doctoral students probably don't think of their dissertation topic this early (I haven't even completed my first course), but I want to be pro-active and work on the thing while taking courses.  So, Rebecca's post on Educational Design Research (EDR) was quite timely.  This isn't my first go around at a dissertation topic, my current topic has evolved over the past couple of years as I was thinking about what I want to do (and which university is best to pursue this).

My initial idea was to blend my background in Instructional Design and MOOCs to teach a language, specifically designing a MOOC to teach Greek as a foreign language to novices. This actually came out of making a MOOC out of my MEd capstone.  This was circa 2011-2012 after my experiences with MobiMOOC, LAK11, CCK11 and before the xMOOCs invaded the scene.  After some thought about this I decided that Greek would be way to much work for a dissertation. There isn't a lot of material out there and available as free OER, or under creative commons, for me to use in designing this course.  If I were to not only design and implement the MOOC, but also a lot of the kickstart materials for it, I might be stuck in dissertation purgatory. So that was scrapped.

Then, I read about the Polytechnic University of Milan switching the language of instruction to English, and I thought that  this would be a good opportunity to refocus the project on native speakers of Italian, in Higher Education, improving their English through a MOOC format.  This had the benefits of having an audience that would need the course (so I wouldn't worry about students joining, in theory), it would be focused in terms of  the target learner (thus cutting down on the variables for any post-dissertation analyses I wanted to conduct and write about), and it gave me an opportunity to brush up on my Italian.  This was when I was thinking about applying to the National & Kapodistrian University of Athens. As we know, I ended up not applying there because there were strikes which prevented me from getting my materials in (and also getting my degrees from the US translated, which required me to go to Greece).  This idea is on the back burner, but I would love to explore it in the future.  I downloaded a lot of CALL and SLA articles during this time frame as an initial literature review which I would love to read and put to use.

Finally, after getting accepted at Athabasca, I thought about who's currently there, what their expertise is, and then further refining what might be a dissertation topic.  It should be stated that my goal with my dissertation is not to do something earth-shattering and wow people with any potential brilliance I might have. The point of the dissertation, for me anyway, is to receive apprenticeship into research, to hone my skills, and to then be certified (by getting my dissertation approved) that I know what I am doing without the training wheels on.  In pursuit of this goal I decided to take the course that I currently teach, INSDSG 684: The Design and Instruction of Online Learning, and make that into a MOOC. This isn't going to be new or novice.  If you look at OLDSMOOC, Learning to Teach Online (UNSW/Coursera) and Teaching Online: Reflections and Practice (Kirkwood CC/Canvas.net) just to name a few (I am sure there are others), you'll see that others are tackling the topic now. But, as I said, I am not interested in novelty, and any research that comes out of those courses will most likely be on the data analysis side (at least for xMOOCs).

So, to address the questions that Rebecca posted on her blog post about this latest incarnation of my topic:

What is your research question? Is it is a ‘design question’?
The thing I wish to tackle with this dissertation is the "conversion" process (even though conversion is not really the right term for this) of a campus course to a MOOC.  This MOOC would need to address the needs of a traditionally online student population paying for the opportunity to learn and be evaluated for 3 graduate credits; as well as a population of professionals out there who need skills, but they are pursuing it more as professional development and don't need (or want to pay) for graduate credits. While I would love to analyze data collected from this experiment in other ways, the dissertation will focus strictly on the ADDIE aspects of the course.

Do enough academics at your institution appreciate ‘design’ as research?
It's hard to say at this point. I have read faculty profiles a couple of time already, but it's hard to really know people until you've talked to them, one way or another.  My instinct tells me that there are enough people at Athabasca University who are interested in design, and considering that this is a "professional"† doctorate, I would think that design research would be interesting to someone.

How will you defend your study to researchers who don’t see ‘design’ as research?
I guess I will cross that bridge when I get to it, but my main take-away point is that all research is designed.  There is a certain know-how and skill required in order to even setup a research design, so design research is really (in a sense) further up-stream.  Furthermore, there is a real need to go back to the established literature of learning (and online learning specifically), design MOOC interventions based on this literature, evaluate and iterate. Otherwise, further downstream it your wonderful data analytics just digital clutter with nothing previous informing it.

How will you differentiate research from practice?
This is also another false dichotomy, in my opinion. You can't separate research from practice in the field of education. I see this with students in courses I've taught (or MEd students I've chatted with outside of the course).  Many of them seem to come into the program wanting concrete answers, absolutes, processes and procedures to be awesome designers, but they don't like research articles that are really focused, or provide caveats and exceptions, and articles that state that "more research is required." They don't seem to get that in order to be good practitioners they need to engage with the research in some fashion and do it continuously. Even the research folks in education can't operate in a vacuum. They need see what's happening in the field so that they can ponder, problematize, hypothesize and test. It's all cyclical, to try to break this into to distinct and separate items is a big issue.


† for what it's worth I dislike the term "professional" doctorate. It sounds like an insult to both those who have worked hard to attain it, and to those who have PhDs, because it makes PhDs sound like they are not professionals. I wonder who came up with this.