This blog has been kindly contributed by Mary Curnock Cook, who is a HEPI Trustee, Chair of Council at the Dyson Institute, Chair of Trustees of the Access Project, a Council member at the Open University and a non-exec Director at the Student Loans Company – as well as the former Chief Executive of UCAS. Mary can be found on Twitter at @MaryCurnockCook.
It doesn’t take a degree in Psychology to figure out that disappointment about exams results will be viscerally exaggerated when those results have been generated by an algorithm. In a normal year, a large proportion of candidates are disappointed with their results, but at least they have an exam script to prove how well or badly they have done.
As if to add insult to injury, the algorithm has also entrenched and exacerbated social inequalities.
The impact of social inequalities on educational attainment is well known and accepted every summer when we count exam achievements and university progression rates, celebrating often vanishingly small reductions in the gaps between rich and poor, north and south, private and state schools. In an extraordinary year when education achievement is derived statistically, these inequalities have been amplified. Yes, schools in more deprived communities are likely to achieve lower average results than their affluent neighbours; but when a computer model generates those results rather than the pupils themselves, it becomes the target of understandable moral outrage. By reflecting reality, at a system level at least, the algorithm has certainly hit a raw nerve by producing better results for independent schools and for pupils in more affluent areas, while the have-nots were delivered damning downgrades against their teachers’ predictions. Add in a hastily designed correction for small class sizes which appears to further favour independent schools and the insult to social justice is complete.
The method adopted by Ofqual failed to look through the lens of individual student performance over three A levels (for example by modelling student level grades in clusters of subjects), it didn’t even attempt to accommodate the maverick and outlier candidates, and it failed to anticipate, let alone encourage, huge demand for appeals. Perhaps worst of all, it was blind to the emotional turmoil that students and their parents would inevitably express when failed by an impersonal, uncaring, ‘algorithm’ that knew nothing about them as individuals.
So what next? As far as I can see, the U-turn horse has bolted for A levels and BTECs because the university admissions process has already progressed too far to be turned back. If fairness is the goal, reverting now to Centre Assessed Grades (CAGs) would require all existing confirmed places to be rescinded and the confirmation and clearing process restarted with a whole set of new grades. This would replace outrage about artificially awarded grades with outrage about university progression, and universities themselves would certainly be unable to accommodate the additional students within the current cap on places, let alone in physical spaces now further constrained by Covid-security requirements.
What is required quickly now is a robust, free and rapidly scaled appeals process. Such a process should include a specific opportunity for schools to identify outliers in performance so that their star performers who have bucked previous trends are not denied their potential to go to university. Capping the number of grades awarded below CAGs might also be an option, as set out below for GCSEs. Clearly, the cap on student recruitment for universities should be relaxed or removed entirely, freeing universities to recruit up to their physical limit. The cap was in any case proposed as a market smoothing mechanism at a time when demand for undergraduate places was expected to be weak for both domestic and international students. Now that demand has increased, such a cap will only serve to thwart student ambition, especially given the A level results anomalies.
More worrying still is the GCSE results that are due to be published this Thursday. This will involve around seven times as many artificially derived grade awards for twice as many students, thus multiplying the potential outrage and political fallout several times over. If it is too late to change the A level results to CAGs, it is also too late to change the GCSE results – different processes for different cohorts would only magnify the injustice already felt by Level 3 students.
There are perhaps some solutions that could mitigate the impending disaster for GCSEs:
- Set out the appeals process before results publication, acknowledge that there will be individual unfairnesses, and encourage students to appeal, preferably knowing that their changed grades will be quickly processed within a given timescale.
- Cap any downgrading at both a subject level and at student level so that no student gets, for example, more than one grade lower than their CAG, and in no more than three or four subjects. (Such a cap on downgrades might be an option for A level students who appeal, although the potential for turmoil in university admissions could make this more problematic.)
- Retain all Grade 4 CAGs for English and mathematics – the difference for students between a Grade 3 and a Grade 4 in these keystone qualifications is often life-defining. No-one deserves to have this decision meted out by an algorithm.
If these approaches were to be adopted, the publication of GCSE results would have to be delayed. This in itself would create significant disruption within schools and colleges trying to manage progression and recruitment for next term. But there are no perfect solutions to the current crisis and I believe most would be prepared to weather yet more disruption to ensure better and fairer outcomes for their students.
With respect, to object to replacing unfairly awarded grades with CAGs on the ground that it would create a practical difficulty for universities is allowing the tail to wag the dog. If the process for awarding grades this year is fundamentally flawed – and the overwhelming consensus is that it is – then it must go. To advocate any kind of appeals process legitimises the basis upon which the grades were arrived at in the first place. If Ofqual’s standardisation process is unfit for purpose it has to be replaced root and branch. If that in turn causes further problems then so be it.
Commentators are all ignoring the other side of this equation. As performance overall has gone up, while we see evidence of massive numbers of students who have been awarded grades that are clearly too low, it is very obvious that larger numbers of students have been awarded grades that are higher than their actual academic standard merits.
Apart from the clear unfairness of this, it also means that universities have no idea whatsoever of the level of academic ability of their year-1 intake. While this was going to be a problem this year anyway, given that many students have done no learning since March, the presence of significant numbers of students with misleadingly high A-level grades is going to lead to heavy demand for academic support when they reach selective universities/courses that have no experience of working with students of their level of ability.
“If Ofqual’s standardisation process is unfit for purpose it has to be replaced root and branch. If that in turn causes further problems then so be it.”
On one hand, you’re correct that if the awarded grades are meaningless then there needs to be a complete recalibration or alternative process. On the other hand, this would have to be decoupled from university admissions as universities cannot safely accommodate the very large number of extra students who would then meet offer conditions as a result.
Unless, of course, the inflated grades of other students are reduced to balance the elevation of artificially low grades. Anyone think Mr Johnson has the will – or political invulnerability – to do that?
University courses aren’t accredited on the basis of entry requirements but on the quality of teaching resource. Of course ‘brighter’ students are more capable of absorbing information more quickly, but the skills acquisition that takes place depends on other attributes. Now is a time of reckoning for many HE providers and those that denied places on the basis of an algorithm should be held accountable further down the line.