On Learning Analytics & Assessment


Yesterday and the day before, the Educause Learning Initiative (ELI) was hosting a spring focus session on learning analytics.  I have to admit that drew me to this talk (in addition to being interested in analytics of course!) was the talk that George Siemens presented at this ELI to kick things off.  The first day was quite productive, but the second day I had too many other commitments to attend to and could not attend for most of it.

One of the back-and-forths that I had on twitter my colleagues was about learning analytics and assessment.  A fellow colleague seemed to be very certain that learning analytics could be used for assessment, and I disagreed.  It's hard to carry on a meaningful debate in 140 characters, so I thought I would write a quick blog post about it. And who knows, perhaps I mis-interpreted what my colleague was saying!

First I think it's good to start off with a few definitions so that we are all on the same page:

Learning Analytics: Learning Analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. (source)

Assessments: Assessments are activities that learners undertake to demonstrate their competence on a (or a set of) course or module learning objectives.  There are various types of assessments, but two most common ones are:

  • Formative: formative assessments are done throughout the duration of the course
  • Summative: summative assessments are done at the conclusion of the course (think: final exam)
The key thing here is that what is assessed is the learner's ability/knowledge after the course. You are NOT assessing the effectiveness of the course.


Assignments: Assignments are activities, that are ungraded, that are meant to help students learn what they need to learn and/or give students adequate opportunities to practice so that they can acquire and perfect those skills.  Assignments might be readings, ungraded tests, and discussion forums.


So, where there was a bone of contention (or at least it seemed like it) on our twitter interactions was around Assessment.  It seems to me that people wanted Learning Analytics to function as an assessment tool. Even though I talked about grading as being fundamental to Assessment, I got back a response that this type of assessment (graded) was Summative, but they were thinking along the lines of formative.

This confused me a little, because it seemed like they were mixing assignment with assessment, then taking this understanding of assessment = assignment, and applying learning analytics of those assignments to see how students were doing in the course. At this point we weren't talking about two terms, but three (and in 140 characters that's not easy)  Furthermore, a couple of days removed from the discussion, it seems to me that this person was probably wanting to assess the effectiveness of the course by using learning analytics, but that (if we go by the definition above) is not the goal of Learning Analytics.

Taking a step back from this, it seems to me that before we (as individuals, institutions, programs) commit to learning analytics, we need to figure out what we want to do. What sort of information do we need?  How can we get this information?  Is this information Learning Analytics (as defined above) or something else?

At the end of the day, I still feel confident that Learning Analytics ≠ Assessments ≠ Assignments

Comments

Post a Comment

Popular posts from this blog

Latour: Third Source of Uncertainty - Objects have agency too!

Image
Continuing on my exploration of ANT, and asynchronous and indirect dialogue with Latour - this blog post will cover the third source of uncertainty, which according to Latour, is that Objects have agency too! As with the previous blog posts, I've pulled out quotes from the book that seemed interesting, or that I reacted to in some way, and I am responding to them here. no tie can be said to be durable and made of social stuff (p. 66) This quote seems to continue Latour's assertion that there is no such thing as "social" or "social stuff" and that "social", or the meaning of needs to be negotiated and better understood.  It also continues the thought that social can only be seen from the actions of its actors, the traces they leave behind, and that these bonds are not durable because they need continuous reinforcement. I guess Social is a perishable item. Left to its own devices, a power relationship that mobilizes nothing but social skil...
Read more

MOOC participation - open door policy and analytics

The other day I was reading ZML Didaktik on the topic of MOOC participants. In MOOCs, one of the big questions is why are people lurking and not participating? If more than 500 people join a MOOC, why are only 10% contributing with any amount of regularity?  On the same blog, in a previous blog post, I had commented (it was an open stream of thought really) that perhaps there should be an open enrollment period, and then if the system sees that certain participants are bellow a threshold of activity, the system may give them the option to self-identify as a lurker, or un-register from the class. This line of thought went along a view that compared to the traditional classroom; in a traditional classroom students register for the course, they can attend classes for a week and then decide whether or not they want to stay with the course or not. If they do stay with the course they know that there is a certain amount of "lurking" that they can get away with, but they do have ...
Read more

You've been punk'd! However, that was an educational experience

Image
It's now the end of Teaching Goes Massive: New Skills Required (aka #massiveteaching) on coursera. Well, almost, we still have a couple of days left. I guess that the lesson here is that we were (the "learners") were punk'd† by Paul-Olivier Dehaye of the Univerisity of Zurich. After that last blog post (and subsequent pickup of the post by George Siemens and others) Inside Higher Education and the Chronicle wanted to chat with me about this MOOC experience. I only had time for one of the two before things went to press, so my 15 minutes of fame went to IHE.  Both IHE and the C hronicle have written about the topic, and have received some information from Coursera on the incident.  Others have also written about it (see here , here and here ) It's surprising to me that the University, and Coursera, waited until after this thing was a big issue in order to respond in the class, and clue people into what was happening. On Wednesday (day 3 of Week 3/3 of the ...
Read more