Sunday, August 27, 2017

Validity...or Trustworhiness?

It's been a crazy few days!  If it weren't for my brother coming down to hangout for a while I probably would have more in common with Nosferatu than a regular human being😹 (having been stuck indoors for most of the weekend).  When I started off this summer I gave myself a deadline to be done with my methods chapter by August 30th (chapter 3 of my proposal).  After reading...and reading...and reading...and re-reading (select articles form EDDE 802), I reached a point of saturation when it comes to methods.  I really wanted to read all of Lincoln & Guba's 1985 book called Naturalistic Inquiry during this round, but it seems like I will just need to focus on specific aspects of the book.

So, in this whirlwind of activity, I went through the preamble to my methods section, my target participant descriptions, my data collection, my data analysis techniques, and any limitations.  I added to these sections, explicated, went more in-depth in each section, I corrected issues that were brought up by Debra in EDDE 805, some outstanding issues and comments from the feedback on the parts I had worked on MDDE 702, and some of the initial comments I got back from my dissertation chair. **phew** That was hard work!  The only parts that I still have left to complete in order to be "done" with my methods section are (1) The ethics section; (2) the validity/reliability/bias section;  (3) a conclusion section for the chapter bringing it all together; (4) an appendix with a sample survey; and (5) an appendix with the participant consent form.  I am considering adding (6) the REB application to an appending as well before I call this section "done".  I am not sure if I will be done by August 30th (as was the original plan) but I think I will be damn close.

That said, there is one thing that is tripping me up, and that has to do with the validity/reliability/bias section.  Bias is actually not that hard.  I think I can write up procedures and things to be on the lookout for in order to avoid bias in both data collection and analysis.  The thing that  is much more concerning is philosophical:  Do I go with Trustworthiness, Credibility, Dependability, and Transferability as what I talk about in this section (coming from the Lincoln and Guba tradition of qualitative research work)?  Or do I choose the more traditional Validity and Reliability and discuss my methods in that frame of reference?  Of course, for qualitative studies the Lincoln & Guba approach makes sense (at least it does to me, and it's references in a variety of other texts I've read on qualitative approaches to research), but at the same time quite a few of the texts that I've read (both on case studies and on qualitative studies in general) still use reliability and validity as terms in qualitative researcher. So, so I "translate" validity and reliability (from the texts) in Lincoln & Guba terms?  Just discuss in the framework of Lincoln & Guba? or try to smash both together?  Perhaps start with Validity & Reliability and transition to L&G terms since they make sense?    I need to re-read Chapter 11 of Naturalistic Inquiry this week to help make up my mind (any thoughts are more than welcome in the comments).

As of this point I am at 23 pages (with 1 paragraph of lorem ipsum text, and quite a few scraps of though patterns for items 1 and 2 above), so I am thinking I should be wrapping this up soon and not getting logorrhea.

Thursday, August 3, 2017

The publication emergency


Paul Prinsloo has a wealth of thought provoking posts on his facebook ;-)  I wasn't planning on blogging until tomorrow, but this got my mental gears moving and thinking (not about my dissertation, but it's thinking nevertheless).  This blog started as a continuation of a comment I left on Paul's facebook feed. The article that got me thinking is an article on the Daily Nous titled The Publication Emergency.

In the article a journal editor (in the field of philosophy) opines (although not with his editor hat on) that graduate students (I guess this means doctoral students) should be barred from publishing until they are done with their degree. He says that this is not a barring of people who don't hold a doctorate, but rather of people who are in process of earning their doctorate.  So, in theory, some with an MA, but not pursuing a doctorate would be welcome to publish their stuff.  So, even if an article is good and has merit, if its author is in process of earning a doctorate they'd get an automatic rejection.

It was an interesting read, and a good thought piece - and in all honesty I think that that since the author is a philosopher he is either trolling us or trying to get us to think more deeply about this topic. I certainly hope so at least because I don't think there is a way for me to disagree more with the position expressed (my disagreement has reached 11 😉 ).

The proposal seems to be trying to address the problem of the proliferation of credentials amogst the professoriate, of which I'd argue that publications is one kind of credential.  And, this proliferration of credentials is making it much more competitive for those in the professoriate at a variety of levels.  To deal with this problem the proposal is to arbitrarily eliminate one segment of the population, barring them from publications. It's arbitrary because someone without a doctorate, as mentioned above, is still NOT barred from publication, except if they are in the middle of earning one. So someone like me publishing prior to entering a doctoral program is fine, but if I enter that gray zone it suddenly is not? (not trying to make this about me, I promise 😛 ).

I see the problem as being more systematic, and the credentialing is a symptom of a larger issue (something we see at level beneath the doctoral level as well!)   Doctoral programs have increased and saturated the market.  Doctoral programs have gone from artisanal experiences, where few people applied, fewer were selected, cohorts were smaller and members shared the 'pain' of the experience. Sort of like going through an elite military program, but in academia.  Nowadays doctoral programs have (mostly) morphed to being cash cows for universities. More PhD† students graduating means more competition in the tenure stream job market; that is of course unless your program was thinking ahead and designed doctoral programs whose goals are not just to create new tenure stream faculty, but also work in other facets of society. More programs means more candidates competing for fewer jobs and resources (journals being  one of those resources). Hence, you need additional credentials to make it, such as doing post-docs (which I'd call exploitative labor), other type free work and volunteering, more papers published, etc., in order to get an edge.  It reminds me a lot of high school when people joined as many extracurriculars in order to pad their college application 😒

So, now you have a problem, my friend! You see, more early academics‡ publishing because current professors claim that they don't have the ability to publish as much because the journal market is saturated - with the author of the article citing 500-600 submissions to a journal per year. Add this to the budding market of predatory journals, and you've got a really bad mix.  An invalid argument that was proposed in the article is "well if the argument of a graduate student is good now, imagine how good it will be in the future!"  I don't buy this.  Academia has setup a perverse set of incentives, including keeping track of citations.  If the goal is to write something good and get citations for it, why wait? Ideas don't have a monogenesis, so if you wait someone else will beat you to the punch. Why handicap younger scholars who should actually be apprenticed into this? Another proposal is to not have anything published before you got your current tenure-stream job not count, so that people can see what you can do on the job.  This also made me roll my eyes a bit. 🙄 Why practice selective amnesia? Simply to sour the milk and prevent people from doing something?

I think academia needs to seriously look systematically at the perverse incentives it has provided over the past 40 years♠ and deal with those rather than just dealing with a symptoms in ways that are arbitrary, and at best, a bandaid on a bleeding patient.  Doctoral students should be encouraged to publish, it's good practice for those who want to go into the professoriate.  Why not have the benefit of mentorship while undertaking this?  Furthermore, the publish-or-perish modus operandi for academic units needs to finally be put to rest.  Faculty positions need much more granularity than what the current system allows for.  Faculty hired as lab scientists should be evaluated more on their research and publication performance. Faculty who do mostly teaching should be evaluated more on that.  Tenure and academic freedom should not just the playground of research PhDs, and we shouldn't try to shoehorn everything into a research PhD framework. This to me seems like a vestige of positivism, and our profession needs to think more seriously about a more dynamic, better representative, professoriate.  More holistic means of evaluation will most likely take the pressure of the publication system.



NOTES
† PhDs is just my short term for any sort of doctorate, I don't really want to get into the whole PhD/EdD thing...
‡ By early academic I don't just mean tenure-track folks, but doctoral students
♠ just an estimate on my part, based on my conversations with others in field more time than I