So here’s a blog post a little different to my usual climate change doom-saying! This week, several colleagues at different universities drew my attention to the growing online discussion about the upgrade of an open-access AI chat application GPT-3 that can now handle complex instructions and produce coherent long-form responses.
People who have tried it out have generated some pretty amazing output including poetry as well as more creative outputs (e.g. an open letter against open letters, and even a biblical verse describing how to remove a peanut butter sandwich from a VCR!). Lots of other cute examples here.
For those of us who teach in universities, the significant implications of this technology are immediately apparent. As a number of US academics have already noted, this could well spell the end of written essay assessment:
So given we’ve just survived another heavy teaching semester (dominated by the ghastly ‘hybrid’ teaching model of combined in-class and online delivery), I thought I’d find out what all the noise was about and give ChatGPT a whirl based on some of my organisational sustainability teaching content. What did I find? Well, frankly I’m surprised how good this technology is!
I started with something easy – a definitional question from an area I’m familiar with:
Not bad, a bit boring and needing more elaboration as a response but basically correct. It misses the critical literature on CE regarding its use as rhetoric and ‘greenwashing’ as well as the way in which CE acts a s a response to the threat of regulation, but a perfectly passable response.
So, I thought, “OK, now I’ll test you a bit” and tried something a little more advanced:
Kind of frightening watching how fast the AI spits out the response and again a pretty decent answer. A more advanced student would define and engage with the concept of ’emotionologies’ as dominant discourses of socially-defined emotional expression, however, if a student provided this with some elaboration and detailed examples/references they would be in the Credit range.
So, I amped it up again and went with a question that would require more specific knowledge and combine insight from two literatures:
Again, the machine spat out a pretty reasonable response. Sure, it missed some of the detailed aspects of Boltanski and Thevenot’s framework which I would expect a student to provide if they had taken one of my courses, but (and it’s a big ‘but’) any student submitting such an answer with appropriate references and examples would certainly pass and with elaboration probably be getting into credit territory. Just imagine what it could do with a more tailored prompt including how many paragraphs you might need (see below)?
So, my conclusion is that this technology has profound implications for university teaching. All of the current anti-plagiarism applications are useless against this as it is being generated by an AI that is not simply replicating previously published material but creating new syntheses of content which are surprisingly good. In short, all online exams and submitted written material can now have been generated at least in part through this type of technology and its not clear how we can assess its origins.
This will require a complete rethink of how we assess our students’ learning beyond the traditional submitted essay and online exam. For instance, some observers have a more sanguine take on these developments and outline how we will just need to adjust our teaching methods and perhaps use technology like ChatGPT as a tool to help students improve their writing and argumentation. However, it remains unclear how the standard essay can remain a viable form of assessment given this new openly-available technology.
Moreover, this technology is not limited to just student essays. As this article by Wharton professor Ethan Mollick in Harvard Business Review makes clear it has the potential to revolutionise all kinds of jobs from writing tasks in businesses like management consulting, finance, advertising and journalism, to code-writing computer applications. Indeed much academic work like developing course syllabus, lectures and research essays could be generated via this technology.
“This is why the world has suddenly changed. The traditional boundaries of jobs have suddenly shifted. Machines can now do tasks that could only be done by highly trained humans. Some valuable skills are no longer useful, and new skills will take their place. And no one really knows what any of this means yet. And keep in mind: This is just one of many models like this that are in the works, from both companies you know, like Google, and others you may not.” (Mollick, 2022).
While there are clear issues around the veracity of the output that can be produced by AI applications, given it will produce what you ask it to produce (e.g. asking it how we know dinosaurs had a functioning civilisation!), the implications for all kinds of work are pretty mind-blowing.
If the explosion in plagiarism, writing mills, COVID, hybrid teaching and general academic work intensification were not enough over the last few years, we now have the reality of AI-based writing to respond to.
As we wind down for the long university break (over the southern hemisphere summer), I wish everyone a happy and relaxing festive season. Rest up comrades, because next year looks like another rocky road ahead!!
Climate, People & Organizations 