Badge MOOC Challenge 3: Competency Frameworks for a Badge Ecosystem

Custom is an odd name of a badge :)

Week 3 of the Mozilla Open Badges MOOC on Coursesites (half way through) and we are continuing our exploration of using badges for the #ESLMOOC. As with previous posts, the prompt of the challenge comes first followed by my thoughts on the subject.


Prompt:
Challenge Assignment 3: Competency Frameworks for a Badge Ecosystem
At the next level of complexity, we consider the ecosystem of five principal sets of stakeholders:

• Learning Providers
• Assessors
• Job Seekers
• Employers
• Standards Organizations

And an additional component:

• Badges
• Competency Definition

Employers are governed by regulations, industry standards, and best practices. Employers need employees whose skills and competencies support and advance business objectives in accordance with these requirements.

Standards organizations get input from academic research and employers about evolving best practices, and, in turn, provide guidance, even governance, over business practices. Standards organizations in some cases define sets of competencies and the types of evidence that are valid for demonstrating these competencies. Badge system collaboration between standards organizations, learning providers, and employers can benefit all stakeholders: When badges are tied to assessments that are themselves aligned to industry standards and best practices, the likelihood of finding the right match between job seeker and employer is greatly improved. And learning providers can use these alignments to offer learning programs that better match employer requirements and offer greater value to their learners.If the ecosystem is in balance in this way, the exchange value of badges is high; when the ecosystem is out of alignment, the value of badges is low.

Based on your analysis of a current ecosystem in the prior challenges, describe how badge competencies could be defined and aligned to improve/revise/rethink the currency exchange within your ecosystem.

Outline a badge system that reflects collaboration between standards organizations, learning providers, and employers.

1. What type or segment of employers will your badge system target?
2. What competencies do these employers value and/or require?
3. How are these competencies defined, and do they need improved definitions?
4. Who maps the competencies badges? Does this mapping include stackable competencies?
5. Is there a badge system that already maps to these competencies that could be applied or revised?
6. Create a persona/archetype that represents a standards organization stakeholder.
7. Write one or more “after badges” user stories depicting the value of your badge system for employer personas.

Brainstorming on Challenge 3

I have to admit that this particular challenge is challenging (no pun intended) because language proficiency is a bit of an intangible at times.  While there are considerations to keep in mind while deveoping testing and evaluation mechanisms (see CEFR manual, pg 11) I would argue that there is no standard that's etched in stone for showing, let's say something like sociolinguistic proficiency. But, on the other hand there probably area ways of being able to infer certain levels of language proficiency in phonological, orthographic and grammatical levels of proficiency. As a matter of fact it seems that most language tests I've been exposed to cover these latter three areas.

So, in thinking about who will be using these  badges, the employer segment of the population, I am thinking that in the case of #ESLMOOC the main users of the badges will be the Universities.  From an employer perspective, if a University is requiring their instructors to teach and evaluate their learners in English, then those members of the teaching staff can earn a variety of badges to showcase their expertise in English.  For new-hires to the department the hiring committee would  most likely take badges into consideration for new employees. For existing members of the Department Personnel Committee, who usually look at scholarship, professional advancement, and merit, would be using badges to evaluate how their existing teaching staff is doing with acquiring and perfecting the language used as a medium for the delivery of teaching.  As an extension to this, I would say that admissions committees in departments could look at badges earned by applicants to their program to evaluate their language skills, and thus to some extend evaluate their ability to succeed in their program.

Without doing a full needs analysis, it would be hard to really nail down what competencies specific universities look for, however since I am working for an academic department I can probably guess at some really high level needs that departments may have in terms of competencies.  In no particular order I would say that department would need their teaching staff to:

• Possess the required academic and professional vocabulary for their discipline,
• Speak English in an un-accented way so that students can understand lectures (especially students who don't share the same cultural background as the instructors),
• Be able to read and comprehend academic literature in English,
• Be able to write concisely and persuasively in English to be able to apply for grants,
• Be able to write academic articles in English,
• Be able to provide written and oral feedback to learners in their classroom in English.

Since language is a fuzzy thing, there aren't really concrete definition for these competencies, at least ones that I can find†. That said, I do believe that, for at least a group of professionals, let's define them as Professors of Education for this example, there can be cross-institutional definitions that are valid for defining these competencies.  For instance, accent reduction is something that employees in call centers undertake, and many actors learn other accents for specific roles in movies.  Since there are parallel professional groups undertaking this specific competency, we could look at what they are doing and how they are defining the attainment of that competency.  Professional vocabulary is something that we can also "easily" assess both through rote method (vocabulary tests), which is unimaginative and a pain, and through embedded methods, like reviewing research and professional articles written by the faculty.  Grammar and orthography can also be tested this way, and it hits on a variety of competencies mentioned above.  To do this effectively we will need to develop rubrics to assist the learners in knowing what and how they need to perform to demonstrate their competence.

As far as who's mapping the badges to competencies, that would be me and the team working on #ESLMOOC.  In all honesty I would prefer to not be the sole mapper of badge-to-competency because I do believe that one person doesn't know everything. I think that a team of MOOC designers would figure out the right mapping of badge-to-competence.  Since the MOOC creation would be a part of my potential PhD process I will have a big part in the design and implementation aspects, but I will be seeking input and feedback on the process.  I do envision the badges to stack-up.  How  much stacking will be possible will depend on the duration of the MOOC and the various opportunities learners take to engage with the material and their peers in the MOOC. At the very least, for some badges, I am thinking of a low and a high badge, where if learners demonstrate B2 level competency during Week 1, but during Week 9 (for example) they move up to C1, the B2 badge would be superseded by the C1 badge.  Depending on the categories of learning addressed by the MOOC there could also be badge collections (not necessarily leveling up) where if learners earn 3 badges from 3 different categories such as "academic writing," "professional writing," and "written student feedback" they may get a "writing maven" badge.   As of now there are no badge systems in this arena that I know of that could be adapted to our use.

The persona for the Standards Organization is kind of hard to think about and nail down. The standards organization, the Council of Europe, is looking (at least from my initial explorations on the subject) at broad generalizations with their CEFR. So, for the purposes of this brainstorming session I will make up what I think is in the mind of someone sitting on these committees coming up with these frameworks.

The name is Radek Zalenka.  Radek sits on the committee that works on the Common European Framework of Reference for Language Learning (CEFR).  Over the past years this working group has come up with the current CEFR.  The current CEFR is designed to be broad enough to be used with any language and to give evaluators general categories that they can use to peg a student's proficiency in.  While there are certain guidelines for developing evaluations, there may need to be some more sub-categories for each of the major proficiency categories. This way specifics like orthography, phonology, and sociocultural factors can be indicated separately. Since languages are different, the badge systems for the different languages may have some common aspects, and some not-so-common aspects with each other. This gives Radek pause to consider because the CEFR is generalized and that's the power behind it.  If badges are created and there is the potential of having different sets of badges for different languages, that can devolve into something completely unwieldy. Thus, Radek is entering this badge "thing" with some skepticism.

As far as "after badges" badges stories go for the employer persona, I can think of targeted support of faculty, and new incoming students to these institutions.  When someone comes in to an institution, either as a student or as a faculty member with some level of  B2, C1 or C2 proficiency that doesn't really tell us what areas they may still need some help with. After all, even some native speakers may have some issues with the register differences in the language used in certain disciplines!  By having learners earn badges for what they know, and how well they can perform, specific workshops can be designed by Professional Development or Faculty Development offices on campus to address those needs.  For example a professor may have a C2 level (Master Proficiency) in English, but have such a thick accent that it renders them incomprehensible at times, but their written elements are just fine.  If this is the case and the learners have not earned the "speak like a native" badge (just sayin'...) then workshops for accent reduction can be organized.  If  graduate students are coming in and they haven't developed that academic language in English yet, workshops can be organized around the attainment of profession-specific language for those students, so they can write and speak in a way that meets with professional expectations.

I see the after-badges aspect mostly are being able to identify target areas that need help in gaining mastery. If you do have the specific badges earned, then you should be all set to be place in the appropriate classes (for students) or the get a job (for teachers).  Your thoughts?




† quick disclaimer, I am only at the beginning of planning this MOOC. I have a ton of materials in my initial literature review, and part of it are the CEFR standards. From an initial quick look, I didn't see concrete definitions, but that doesn't mean that they aren't there.