This HEPI blog was kindly authored by Bess Brennan, Chief of University Partnerships at Cadmus.
Post pandemic, how can universities maintain the benefits of digital delivery of exams while preserving academic integrity in an online environment – and do it at scale? It’s a serious challenge many institutions have been facing.
While the pandemic accelerated the digital delivery of assessments, it also presented academic integrity issues. These concerns have only increased with the emergence of Generative AI. The alternatives – digitally proctored or invigilated exams – come with their own drawbacks, from impinging on the student experience to challenges around student privacy and a lack of digital features. They are also built solely to guard against cheating rather than supporting student success and outcomes.
In the face of these unappealing options, many institutions have been contemplating an equally unappealing choice: to return to on-site, paper-based exams, with all the expense, laborious work for academics and lack of benefits for students that this entails.
Yet, there is another way. The University of Melbourne is implementing on-campus digital exams at a mass scale with new co-designed software that supports student success and has the highest level of academic integrity.
Co-creating a solution
How are they doing it? Having previously experimented with Bring Your Own Device (BYOD) assessments but facing new concerns around security, and keen to maintain the benefits of the Cadmus platform students and staff were used to, Melbourne and Cadmus worked together on an answer.
The solution uses the baseline product already in place at the university, which offers a supportive, end-to-end online assessment workflow, and adds a layer of security through integrated locked browser capabilities and student working analytics. These monitor the process around how students are constructing their exams to ensure an authentic process, flagging unusual behaviour in the exam hall for educators while students are in the sitting.
How does it work in practice at Melbourne?
In the first semester of 2024, the University of Melbourne ran nearly 50,000 on-campus written exams (around half the number that were run pre-pandemic), of which nearly all are now scanned, plus another 38,600 implemented digitally. That number is expected to rise in the semesters that follow. Part of that increase will be through hybrid digital/paper exams. These are being trialled for those exams currently on paper where work includes hand-created visual representation, such as Mathematics, Music and some foreign languages. In these cases, only those questions that need to be handwritten and then scanned in a digitised paper workflow will be, while the rest of the paper can be completed as digital, within the same sitting.
In the 100% digital exams, all answers are submitted via Melbourne’s Learning Management System (LMS) and students use their own laptops. Inevitably, given the sheer numbers involved, there tend to be a handful of issues with student devices during each sitting but the exam hall is equipped with a fleet of laptops for direct swap outs for any student who has a device problem, rather than exam hall staff trying to troubleshoot personal laptop issues on the day. Students can also pre-book a university laptop if they prefer.
A benefit of the Cadmus platform is that it saves students’ work every three seconds, so if there needs to be a device swap out during an exam, a student can log back in within 30 seconds and pick up where they left off, able to see what they have written up to that point.
Some exams require additional resources to be provided to students, such as court cases or formula sheets. Melbourne is currently providing these as hard copies on the desk so students can refer to them while answering the question digitally. In this way they avoid having to jump between different screens on their laptops while taking the exam.
For subjects that require large and complex datasets, often in the form of Excel spreadsheets, or specialist software, the university is exploring providing locked-down fleet computers that can only access certain applications.
The transition
The introduction of on-campus digital exams was a major change project for the university, which had to prepare stakeholders, educators and students alike with roadshows, training and drop-in sessions. The exams themselves are run by the university’s event teams, who treat them as a massive event akin to graduation ceremonies, with support for the assessment design and delivery from the Teaching & Learning Innovation and IT teams
And exams certainly are a massive event – the primary exam venue seats 3,000, with two or three sessions a day for three weeks, and 1,500 of these seats were digitally enabled for semester one in 2024, rising to 1,800 in semester two and targeting eventually all 3000. A further 370 seats are digitally enabled in an alternative venue, plus 2,500 places for students with special needs over the three-week period in bespoke configured venues.
Preparing the exam halls was, in itself, a huge task, involving fibre being cabled into the historic building in which the exams are held, with wifi access points hanging from the rafters and galleries. Every desk is kitted out with a power strip and a digital exam checklist.
The benefits
There is a cost to scanning the digital papers but also a significant return on investment in the form of workflow savings. Previously hidden costs in faculties of tutors splitting out the paper exam script books, separating them out into questions for different markers, have gone and faculties moved from having dozens of tutors marking to only a handful, or the subject coordinators marking everything themselves.
Another major change is the opportunity for soft rubrics, where rubrics can be easily altered if students are answering questions in different ways, thereby improving quality assurance and baseline marking. An unexpected benefit of the digital marking of scanned papers was the ability to mark remotely, freeing up a previously constrained time of year, with some academics choosing to do so while also travelling to northern mid-year conferences.
From Melbourne to Manchester
Having leant on Melbourne’s approach to using Cadmus’ secure solution for exams, the University of Manchester, which has been piloting Cadmus for two years to deliver open book assessments, is also preparing for a major rollout of Cadmus-enabled in-place digital exams to mobilise the university’s strategy of delivering all assessments digitally in the next two years.
Manchester is responding to a logistical need for more assessments in repurposed spaces, particularly for the medical school where assessment is highly regulated. But it is also part of a wider, strategic approach towards assessment based on three basic principles: all assessment should be relevant with real-world application where possible, it should be inclusive and it should be trustworthy.