ChatGPT: Reflecting on a semester of use and miss-use in a writing class
What a semester it has been. In January I used ChatGPT for the first time and was immediately reeling from the implications for University education broadly and my writing course on Freedom of Speech in particular. I committed to radical transparency and decided to show my students how to use the tool and be open to their utilization in exchange for their honesty with me. I have chronicled the results on a blog my colleague Nik Janos runs, but as the semester draws to a close I wanted to reflect on the experience for my own sake and in the hope it might be useful to others.
First, students had almost no knowledge of ChatGPT or other AI programs before the semester started and were understandably surprised. This changed radically over the course of the semester partially because of my own integration and in part because of an emerging conversation on our campus and in the broader world. Takeaway: this is the most rapid diffusion of innovation I have ever witnessed with many people going from complete ignorance to regular users in weeks. This will accelerate again as AI generation tools are embedded in office suite products.
Second, utilization is much more diverse than simply having the program write responses and essays for you. Complete replacement of writing was the initial concern a lot of us had when we first encountered the program, but student use was surprisingly varied. By the end of the semester utilization ranged from idea generation to overcoming writer's block, to creating counter-arguments. This unpredictable and varied use mirrors the kinds of things we saw from the recent Chronicle of Higher Education piece. Takeaway: with a tool this powerful students (and people broadly) will continue to find more use cases and creative implementations.
Third, we were right about the arms race metaphor. Nik and I originally proposed that professors in the “fight” camp committed to working against AI would be in a perpetual arms race as the programs developed. Just in the past three months we are seeing several AI detection programs (check out our new Youtube series for a full breakdown in the coming weeks) rushed into use with varying degrees of viability. Takeaway: this will accelerate and diversify for different programs. Accrediting bodies will start to weigh in on program standards in some disciplines and mandate a “fight” disposition. Other programs will work with the program and campus wide policy will be extremely difficult to make.
Fourth, bad faith utilization is absolutely a thing. While I am no fan of the turn-it-in system of AI detection, it did indicate several student papers were written by AI and in one case a student had even left in the “Regenerate Response” text from the bottom of a ChatGPT output in their short answer. The radical transparency posture worked well for gathering information. It did not dissuade students from utilizing AI and just not telling me about it in several instances that were quite obvious for both turn-it-in and because of my own fluency in free speech literature. Takeaway: we have to develop a posture to dissuade students from using this to create whole assignments and passing them off as their own work. I don’t know what that is yet, but this is a real problem I am acknowledging.
In our personal experimentation with ChatGPT we kept being surprised by the capacity of the program. Its ability to write from different perspectives and for different audiences was unexpected. This also complicates a common refrain we have heard from colleagues about asking students to write from personal experience as a way to circumvent AI. We have also spent time in each “camp” we initially theorized. We have also found the thirst for information about this new area to be incredible. Nik and I have been successful, but we are not world-renowned professors in our disciplines, let alone in artificial intelligence. Nevertheless one or both of us have helped with a podcast and several webinars, including a recent collaboration with Alchemy.
The refrain “if I’d only known then what I know now I would have…” often pops into my head when thinking about this semester of experimentation. Honestly, I don’t know what I would have done differently. I will continue to write, read, and reflect this summer as I try to develop some policies for the Fall. Right now, I think the disruption to education is much bigger than I anticipated. Issues like “is it plagiarism?” or “should we tell them about it?” seem quaint. I think we are headed toward bigger questions like “why are we doing what we are doing?” and “does the world we were preparing students for exist anymore?” seem more appropriate. I’m not sure I’m ready to answer these questions, but I know I don’t have a choice. As a wise man once noted, the times they are a-changin.