Assessment and AI
Artificial Intelligence represents a radical challenge to doing meaningful assessment in higher education and in education broadly. Turning the lens of inquiry back on our own classrooms has always been complicated, but we have never faced a disruption like this before.
The argument here is fairly simple. We now live in a world where student work is often indistinguishable from work aided by, or created entirely by Artificial Intelligence. OpenAI, the company which made ChatGPT and Dall-E formally admitted that detection software does not work, confirming what we have been writing about for a while. Traditional assessment of a learning outcome takes a product from a student, the completion of an exam, writing, or some representative anecdote and evaluates whether the work product illustrates student mastery of a knowledge base or skill. So, if we cannot tell what students have produced compared to what AI has produced or co-authored we are no longer, as of right now, assessing student learning. We are instead assessing some combination of student learning and proficient AI usage.
This has several implications ranging from classroom practices to institutional accreditation.
If you are in the classroom and relying on assessments to determine if you are teaching well you cannot continue with business as usual. You may need to reevaluate the viability of your learning outcomes or assignments.
If you are working on a cumulative report for a program or an institution, you may need to draw a line between assessment conducted prior to November 2022 and after. The launch of ChatGPT last year means you cannot have confidence in written material being reflective of student understanding.
If you are an accrediting body for a discipline or program–time to rethink what you are doing. We need new guidelines from organizations with close connections between government, industry, and education.
One thing we cannot do is give up. We have come a long way from assuming students were learning because we were speaking and administering exams. This is a disruption, but it is one we can navigate together.