Appropriateness of primary materials? Thoughts on peer review
Anyway, there was some feedback, though, that I take issue with. In order to do this initial typology our literature review spanned over 100 sources ranging from academic, peer reviewed articles, to conference presentations, to news items from Inside Higher Ed and the Chronicle, as well as blog posts from respected leaders in the field, such as Stephen Downes, George Siemens, and people who've run their own MOOCs. We opted to only use academic literature that was free on the web in an open access format. I do have access to library research databases, however I find it very odd, philosophically, to study open education and to rely on so many closed access journals in order to do my research. I could definitely see the peer reviewer's point that we should go after more (closed access) academic articles to increase the pool of our data, however I take serious issue with the following advice that we got:
The collected materials in this manuscript are not high quality document owing that most of them are informal documents. The authors should collect academic high quality papers to survey and analyze.Just because something is not in an academic, peer-reviewed, journal doesn't mean that it does not have some sort of quality, and it does not deserve attention. There have been quite a few blog posts out there that were pre-cursors to academic articles that someone in an academic article down the road deemed to have sufficient quality to publish. Furthermore, if someone has published in peer reviewed articles, and regularly gives conference presentations and keynotes, don't their blog posts carry more weight than mine or someone who's a novice in the subject? And, last but not least, what happens in professional publications such as the Chronicle and IHE never makes it to academic publishing (see for example the issues with FOE MOOC and #massiveteaching). These things need accounting and addressing, and if they never make it to publication, for one reason or another, they become invisible to educational research. This seems like a big failure to me.
Another thing that piqued my interest is the idea of laying down the groundwork. I understand that there needs to be some level of definition of terms in academic research, and when I eventually do a dissertation I expect that I will be defining a lot of terms before I get to the heart of my own experiments. That said, in an academic article, which usually has a limit on the amount of words you can put in it, how much space do you use for definition of terms, and how much do you devote to the heart of the matter? How reasonable is it to expect that, in the age of Google and in light of your providing of a full reference list, that you don't necessarily need to briefly describe terms like xMOOC, cMOOC, CCK, and PLENK? At the end of the day, I am not averse to defining such terms (in addition to any existing citations) but I do wonder how this is counted against the authors of papers when there are defined word limits for articles?
Your thoughts? Any other peculiarities of peer review you have a bone to pick with? :)
Image from: http://www.phdcomics.com/comics/archive/phd081508s.gif
Comments
The extent to which terms need to be defined, I think, depends on the expected audience. Amongst people who are engaged in research around MOOCs, it is reasonable to expect c- and x-MOOCs to be common knowledge. CCK and PLENK might need a brief description for those without enough background in the history. I wonder if citing and explaining things that are common knowledge doesn't diminish author credibility.
My two cents anyway.
1. Should this type of writing be more accessible to other professionals who are not perhaps completely involved in the topic, without a jargon dictionary?
2. Would this article be digital? Couldn't some excellent hypertextual footnoting help here? I mean… we have the technology…
"Physical" access in that many articles are behind paywalls and are expensive to access (some are up to $30 per article! Insane!)
"cognitive" access (for lack of a better word) because readers have not necessarily been apprenticed into the lingo of the discipline.
Part of the problem, in my opinion, is the issue of tenure and promotion in higher education. A professor needs to publish in high impact journals (which are exclusionary, in many senses of the word) in order to have that cache, and get to keep their job. Of course, this has the effect of keeping people out. Pretty silly if you ask me. In order for the system to change, however, you need people on the inside willing to take a risk of not getting tenure by experimenting with open publishing (open in many different ways). Of course if one person does it he/she is a target. If many people do it, there is power there! :)