I knows it when I sees it

When I first started teaching graduate courses back in 2012 (wow...time flies), my hallmark course was titled The Design and Instruction of Online Courses.  Most of my students were folks who were learning to be instructional designers, or faculty who wanted to learn to teach online and had some years of teaching in person under their belt.  One of the most common things I heard in that class was that it was easier when teaching on campus to determine if students were engaged or paying attention.  Just by looking at them, you could determine this by their facial expressions and body language. Eh...OK🤷‍♂️.  While I don't disagree that some people may have this ability some of the time with some of the students in front of them, I still think that people tend to overestimate their ability to do this.

Fast forward to today, browsing through the r/Professors subreddit, one tends to find posts asking fellow redditors how they can tell if their student's work is generated by an LLM. We all know (or should know) AI detectors are snakeoil and don't work; so faculty are left with reading the tea leaves, guessing, and injecting their own gut feeling - and I would argue subconscious bias.

I personally find the guessing game an exercise in futility.  I remember being in college and reviewing peer submissions in a 102 class and thinking: "Did you write this? Did someone else write this?  Did you write this with a thesaurus next to you?" This type of second-guessing isn't new in the LLM era. I think the LLM-era has made us more paranoid, and much more eager to suss out those "stinkin' lil' cheaters."🙄

Now, don't get me wrong, last summer I taught a course, everyone was a grownass adult, a faculty member of many years, and I still got submissions that seemed AI-generated. I rolled my eyes whenever I read these.  They weren't particularly great, but they were fine enough to pass.  I gave feedback anyway as if people actually wrote these submissions.  Why? There was no concrete proof that AI was used, and if it was, I don't know if students modified/edited that output to better suit their project, keeping what worked and discarding what didn't.  It made for a boring read, but I wasn't giving points for originality and intrigue. 

My feeling, after discussing LLMs this past year in the context of cheating, is that many faculty want some external policeman (or bogeyman) to enforce any sort of AI policy that keeps things like they were in 2019. In the absence of that, we have pinky swears and attestations that student work submitted isn't AI-derived (😹), and then we start looking for the AI bogeyman whenever we read something that seems slightly off.  It may be AI...or it might be students faking it until they make it...