This HEPI blog was kindly authored by Diana Laurillard, Professor of Learning with Digital Technology at UCL Knowledge Lab.
This blog was written after Professor Laurillard attended a HEPI roundtable dinner hosted by Studiosity. Many of Professor Laurillard’s reflections are in parallel to the topics discussed during the insightful roundtable discussion, including the use of AI for learning gain as opposed to productivity, for the proven student outcomes following immediate formative feedback, and for collaboratively supporting over-stretched staff.
If the AI community had ever asked teachers in higher education what we need from them, we would very definitely not have asked for a technology that makes it easy for students to deliver second-rate essays with a lot of factual errors. That is what they provided – unleashing a desperate collective action by teachers now having to learn for themselves, and then teach students, how to use these deficient tools, and develop the skills that will probably be redundant within just a few years.
Whether they are training and using a generative chatbot, or countering its bad effects by moving to more labour-intensive forms of forms of assignment, both activities increase teachers’ already unmanageable workloads. They will welcome advice and guidance from the AI consultants suddenly emerging. Even that creates a significant time cost.
If the AI community began instead with what we need to improve student learning, then feedback becomes the focus, and there is a lot that AI could do. Feedback is fundamental to the learning process because it links our actions to our intended goals – in formal and informal learning.
How could Generative AI (GenAI) assist learning?
Large language models (LLMs) that respond to prompt questions have been trained on massive datasets of existing texts. They use pattern recognition techniques to develop a network that uses the prompt to generate the most likely answer, or the best match, to the data it has analysed. The phrase ‘stochastic parrot’ describes it very well: a purely probabilistic deduction from known texts.
Fortunately, students quickly realised that they cannot generate a good essay this way, and teachers are being careful to warn them about this. But what could it usefully do?
One key problem with feedback in higher education is that teacher feedback on a submitted assignment is never immediate. It often comes so far after the submission, days or even weeks after, that for the student it has become irrelevant. And there is nothing they can do to improve the quality of the previous one. With this in mind, imagine the value of AI-generated feedback, once the essay is drafted. But that only works if there is a student-generated draft in the first place.
Higher education already uses formative feedback methods, so one approach is to start there and adapt. Peer review is a useful exemplar that could be adapted to use GenAI as the ‘peer reviewer’, where:
- the teacher devises a rubric to guide students’ submissions
- the student submits a draft to the virtual learning environment (VLE)
- each student prompts their AI peer to create and submit a draft
- the VLE orchestrates the peer review process: each student gives feedback to their AI peer in terms of the rubric, each student prompts their AI peer to give feedback on their own draft, each student revises their submission according to both the feedback from the AI peer AND the feedback they gave to the AI, and then the AI is invited to comment on the quality of the review it receives.
Students benefit most from doing the review, in addition to receiving feedback, and can then submit an improved assignment, which benefits the teacher by reducing their work on the final feedback.
Human students are tolerated by students as reviewers, not so much as evaluators, precisely because they are peers, although peer feedback is often valued. Some worry about exposing their work to peers, others about them stealing their ideas. So AI as a peer could be preferable.
The primary learning value is in doing the peer review and in the immediacy of the feedback. The secondary value is in the ideas prompted by the reviewer, and AI should do well at this. Overall, the learning value is that the immediate interactive process motivates students to learn through actively producing a draft, two reviews, and a revised draft. They also learn through experience how to devise productive prompts to the GenAI.
Downsides
There are three significant costs of using GenAI.
The cost for both students and teachers is high. Students get too much help, so don’t need to engage, and receive inaccurate guidance and information, leading to poorer learning outcomes. There is an increased workload for teachers in either training the bot, or using one, or countering the bad effects, or working with AIEd (AI in education) providers, or moving to more labour-intensive forms of assignment.
Secondly, the opportunity cost of the current focus on what GenAI can offer reduces funding for research and development (R&D) on the more elementary and very useful AI models, such as the classic program. A program is a series of if-then-else statements, with rules programmed explicitly by a data scientist, to provide a rule-based, symbolic, explicit program of actions, for example, concept-development games, role-play decision-making games, interactive models of the worlds of maths, science, engineering, finance, business, and so on, for testing and exploration. Many other useful tools and applications for learning that deserve much more R&D use quite standard computing – and with much more sophisticated use of pedagogic design.
Thirdly, the energy cost of this form of AI is a large multiplier of current comparable methods of support. The cost of training the models is very high: for GPT3 it was 1.3million kWh. This is equivalent to the energy cost of 16 person-years (the annual per-person energy cost in the US is about 80,000 kWh). And the cost of using the models is massive: one question to Google costs 0.0003 kWh of energy. One prompt to ChatGPT costs 0.1kWh, that is, 330 times as much for every student transaction.
And yet millions of teachers across the world are being guided towards asking all their students to engage with this.
Conclusions
Don’t encourage the use of GenAI for students’ text-based assignments. Instead, direct that funding and creative energy towards better-designed versions of existing digital education tools.
Do encourage teachers to collaborate on innovative ways of using this new technique among the many other digital tools to promote engagement with understanding and guide social learning.
Studiosity provides students with equal-access to immediate, formative, personalised writing feedback, study support, and peer connection at scale. Studiosity’s AI technology is built for learning, not for generating outputs, with a closed-system AI training to higher education privacy standards, ‘humans in the loop’ and quality assurances.
I agree entirely with your analysis and conclusions, especially the value of student review and peer review. The one additional area I have seen work well is the use of AI by educators to improve the quality of feedback they provide. Given the scores we receive on quality of feedback generally, running feedback through GenAI against a quality rubric (SMART) can make feedback more useful.