AI has changed your class. Panic: No. Re-Build: Yes.
When I was taking stock of what I learned with students and Artificial Intelligence (AI) this semester I was at a loss for how to move forward in the Fall. It was obvious a more systematic approach was needed so I started to develop a framework to evaluate the impact of AI on different objectives and assignments. I thought it might be useful for others so I am releasing it here with explanations for each column and a partial example based on the public speaking class I coordinate. I think this represents the most significant disruption to higher education in a generation and we need to be re-evaluating almost everything we do. This is one part of that puzzle and I hope it is useful for others. Nik and I are planning to follow up with a more comprehensive guide to pathways for changing assignments and possibly Student Learning Outcomes (SLOs).
This is a framework for evaluating the impact of AI programs on existing course SLOs and correlated assignments. Use this framework to figure out where AI tools will disrupt your existing course and get some direction on possible solutions. In the table below (download a blank table here) you will see an example of the framework filled out for CMST 131, speech communication fundamentals. The SLOs have been abbreviated, but they are available in full at the bottom of this document. In this example the subpoints of the SLOs have not been included for the purpose of clarity, but in a full-analysis they should be included.
Student Learning Outcomes: This should be taken from your course as it is. Don’t worry about what will change, just break these out into the framework. If there are subpoints, make a decision about whether they will need their own row.
Related assignments or activity: Your SLOs should already be lined up with key course assignments. If you don’t have material in your course to support the SLO or if you have a bunch of work that does not match up with an SLO, that is its own problem.
AI disruption: This should be in relation to the assignments. Don’t worry about whether tools can check for AI written work or anything like that, just answer the question: Can one of the AI programs do this assignment in whole or in part? Overtime, returning to the framework will be important. Right now the tools we know about can do a lot including making outlines, researching ideas, answering multiple choice exams, creating slide decks/visuals, writing code, creating websites, and even writing from particular perspectives with adequate prompting. You may need to crowdsource some of this through a listserv or message board as it is seemingly impossible to stay abreast of all the new tools. Keep in mind there are a wide range of tools not just ChatGPT and the goalposts move on this all the time so what might be a “2” today may turn into a “4” in a few months.
Should the SLOs change: This gets to some fundamental questions on what we are preparing students to do. Sometimes it is a conceptual building block for a future course or a particular skill they will need in public life or in a job. Regardless, we need to ask whether the world we were preparing students for still exists.
Take outline production in public speaking as an example. Learning basic principles of organization like intro-body-conclusion will still be important, but the generative aspect of this work is much more likely to be outsourced to an AI program than done manually by people in professional spaces. For now, the most likely scenario will be to have an AI program draft some points and then have a human with knowledge of a subject area evaluate and customize the output. In this case, the SLO probably needs to change to reflect a new reality.
Should the assignment change: The SLO may also still need to be intact, but AI may have disrupted the correlated assignments. In this case, the assignment needs to change. Solutions for this will vary, but may include: AI detection technology like turn-it-in, or GPTZero, in class work away from AI tools, adjusting expectations for outputs, or working with the existing tools in a more integrative manner. As I mentioned earlier, Nik and I are working on a follow-up with some specific options and ideas. Some of this may be driven by external accrediting bodies or institutions.
Take the example of quizzes in public speaking which we have used to evaluate knowledge of key subject matter areas. We may have to return to doing quizzes in person in class to evaluate understanding or develop an alternative mechanism..
At the conclusion of this process you have a map for what needs to change. The nature of that change like specific SLO adjustments or assignment reconfiguration is beyond the scope of this framework and will require intensive customized work, but you now have an idea of the work you will need to do.
CMST 131 Learning Objectives
By reading assigned material, watching video lessons, attending and participating in labs, and successfully completing each assignment, by the end of this course you should be able to:
Effectively evaluate content for oral presentations
By researching sufficient types and sources of content
By selecting relevant and appropriate content for a specific purpose and audience
By using content ethically by orally citing sources
Effectively organize content used in oral presentations
By clearly identifying the thesis of the presentation
By demonstrating the connection between the content and the thesis
By presenting the content in a logical and meaningful order
By demonstrating creativity in the development of content through the use of themes, metaphors, larger organizational principles, etc.
Effectively deliver oral presentations
By employing a style of delivery appropriate to the context
By demonstrating confidence with selected style of delivery
By utilizing nonverbal channels to enhance delivery (e.g., eye contact, facial expression, gesture, body movement, paralinguistic cues)
By demonstrating responsiveness to audience as necessary (i.e., adjust language, provide clarification, etc.)
Demonstrate effective reading and writing skills
By completing online quizzes
By constructing speech outlines according to APA style
By completing online activities
Demonstrate effective listening skills
By accurately summarizing speaker content (or main ideas) as an audience member
By providing constructive feedback for speakers as an audience member
By providing relevant contributions (i.e., information, opinion, and/or argument) in various settings (i.e., group discussion, meeting, public forum)
Demonstrate understanding of how context influences public speaking
By defining speech purposes and goals for particular speech occasions
By identifying factors relevant to effective oral presentations for specific occasions
By selecting topics that are relevant to particular audiences
By tailoring speech content to respond to the psychological, social, and/or cultural needs of specific audiences