ChatGPT: I Told You So!

"I told you so" is incredibly satisfyingly to say and an unproductive way to engage in dialogue. In early conversations about ChatGPT with other University professors I was often told of limitations like "the prose is clunky" and "this is full of inaccuracies that are easy to spot for experts" and an even less specific, "this won't effect the way I teach and grade" often implying an imagined level of sophistication that would make them immune from the technology. I often replied that some of these things may be true now, but they won't be true 18 months from now with more sophisticated iterations of the technology. Turns out I was significantly wrong about the speed of technology. Four months after the initial rollout ChatGPT launched version 4.0 and last night I paid to upgrade my account to check it out.

Some of the inaccuracies persist when you ask the program to write about something incredibly specific. For instance, I asked it to write a bio of my collaborator Nik Janos and either learned he is living a double-life or that the program continues to conflate information it finds in similar spaces.

However, the writing, research, and integration is surprisingly improved. Previously we have mourned the death of the quick-write as an effective mechanism for assessing student understanding in our first-response article. Now I find myself reconsidering the viability of most writing.

Recently I graded paper proposals for my writing course on Freedom of Speech. I wondered if the new version would produce a viable paper proposal which the previous version struggled with. The results were staggering and I have copied the legacy response (first) and the new response (second) into the same document if you would like to check out how the program thinks we need to revise section 230 of the Communication Decency Act. The writing is better, the research is excellent, and it is less distinguishable from human writing (which of course it is drawing on). The format is not what I ask students to do, but editing this to fit my expectations would take about 5 minutes on top of the 1 minute it took to generate the response.

The new version of the program appears capable of doing quite well on graduate exams, the SAT, etc. so perhaps I should not have been surprised at the progress, but I was.

It is difficult to image what this and similar tools will be capable of in six months or six years. For now, I plan on continuing to work with the technology in my class and show students the new capabilities on Wednesday when we meet. I also plan on calling colleagues, screaming "I told you so!" and then hanging up. As a professor in Communication Studies, I can assure you it is incredibly effective.

Don't forget to check out our full-website. Nik wrote a great article about the reactions of his sociology class recently and we will be uploading a response to a recent VICE article along with some analysis of academic honesty issues in the coming weeks.