- This HEPI blog was kindly authored by Rob Cuthbert of the Practical Academics consultancy.
Let us assume that higher education institutions want to have the best students they can get, admitted in the fairest way possible: not a very controversial assumption, even if sometimes people look to satisfice, by settling for something good enough, but perhaps not best. The assessment of ability and potential to succeed in most English universities is based to an important extent on the grades achieved at A-level, complemented by other factors whose importance varies with context. Universities use grades, often in a fine-grained way, to set conditions for admission and then to discriminate between applicants of apparently similar standing, perhaps driving school attainment but certainly helping the universities to manage their admissions to minimise or prevent over-recruitment.
Research by Dennis Sherwood (independent), discussed in a recent HEPI blog has shown that grades for GCSE and A-level examinations are unreliable: one grade in four is wrong. However good the marking of exams may be, there is inherent ‘fuzziness’ in all subjects, which has led Ofqual to admit that ‘more than one grade could well be a legitimate reflection of a student’s performance and they would both be a sound estimate of that student’s ability at that point in time based on the available evidence from the assessment they have undertaken.’ (Ofqual, 11 August 2019). Michelle Meadows (Oxford), formerly Ofqual’s Executive Director for Strategy, Risk and Research, in evidence to a House of Lords Committee on 30 March 2023, said: ‘It’s really important that people don’t put too much weight on any individual grade.” Ofqual head Glenys Stacey said in 2020: “It is interesting how much faith we put in examination and the grade that comes out of that. We know from research, as I think Michelle mentioned, that we have faith in them, but they are reliable to one grade either way.’ (Dame Glenys Stacey, 2 September 2020, Q1059)
Sherwood’s analysis seems irrefutable, and it should have profound consequences for admissions. Prized university places may hinge on one difference in one grade, but all grades are only ‘reliable to one grade either way’. If one grade in four is wrong, the chance of one student getting three correct grades is on average 27/64. However the average conceals wide variation, with STEM subjects less ‘fuzzy’, whereas in English, for example, only about half of all grades are correct. Ofqual has repeatedly rejected Sherwood’s analysis and indulged in ad hominem attacks, without refuting his arguments. Worse, Ofqual has also reduced the possibility of successful appeals against grades to a negligible level. Changes to grading practices to deal with the unreliability are feasible and might even be straightforward, but Ofqual has shown itself unwilling to consider such changes. It has welcomed the reversion to pre-Covid grade distributions, but that makes no difference to the underlying unreliability of grades. It follows that higher education institutions must do what they can to ameliorate the unfairness which inevitably follows. How should admissions practices and criteria change?
There is a range of possibilities. First and simplest, albeit broad-brush, would be to accept the reality of unreliable grades by announcing that all applicants will be considered in terms of their grades plus one grade. A BBC applicant would be considered as AAB, and so on. (This would be a more drastic response than Sherwood says is necessary, but Ofqual refuse to make the necessary lesser changes.) That change in admissions practices would give all the benefit of the doubt to applicants (save for a tiny minority who might have merited two grades better). It would be fairer to allow the possibility of grades being lower as well as higher: this is also achievable. For example, adopt a formula which allows any one grade to be increased by one, and any other grade to be reduced by one, with the third unchanged. Another approach would be to accept grade changes for only the ‘fuzziest’ subjects (such as English and History). For example, simply consider grades plus one for a defined list of subjects, or all subjects except STEM. There are of course many other possible formulae, but the simplest, and least likely to lead to objection or complaint, is the first. It would overstate likely achievement, and it would add to congestion at the upper end of the range of grades, but it would ultimately be fairer than the lottery of the existing grading system.
The unavoidable consequence would be compliance with Michelle Meadows’s recommendation not to put too much weight on any one grade. This would cost universities more. Admissions officers would be bound to give more weight to other considerations, such as universities’ own admissions tests, the UCAS personal statement and more. Interviews might become more widespread. Universities would have to invest more in the selection process, while perhaps reducing their marketing spend in other directions. There would also be logistical complications. The timescale for considering applications might have to be extended, perhaps to match the longer timescale already in operation for applications for Medicine courses, and for Oxbridge. These are significant issues with resource implications, but the alternative is to continue in the unfair but convenient fiction that grades as awarded are an accurate reflection of performance, ability and potential.
Would this change be fairer? For some courses at some institutions, applicants are even now admitted after results day or in Clearing with lower grades than at first required. To a considerable extent, institutions might cope simply by adjusting grade requirements, but would now be less concerned that the lottery of the existing grading system had eliminated promising applicants. The highest grades would be more widespread; the long-term effect is likely to be fairer to most disadvantaged applicants – they might be more inclined to aim high, and they might be more likely to pass a grade-linked threshold for consideration. The change might reduce the need for contextual offers, since many such offers would not allow for more than one grade below the standard offer. All institutions would be driven to give more weight to other information about applicants, and perhaps to rely more heavily, for example, on partnerships with schools and colleges, which will be very well-informed about the potential of their higher education applicants and their ability to succeed. Professor Stephen Jones (Manchester) and his colleagues have recently argued for changes to the UCAS personal statement because their research suggests it is unfairly discriminatory, especially against applicants from disadvantaged backgrounds and / or less supportive schools. The jury may still be out on some points, but UCAS have shown their willingness to consider changes, and consulted on which changes to make.
What of applicants who do not rely on A-level grades? Evidence on the grade reliability of Scottish highers, the International Baccalaureate and the range of BTEC and newer technical qualifications is much less clear-cut, so this would be problematic, but it might be safer (or fairer) at first to allow a similar formulaic dispensation for those qualifications too.
There is plenty of evidence that some categories of student may suffer disadvantage in admissions because of the grades they achieve, if for example they come from some ethnic minorities, if they have lower socioeconomic status, if they have disabilities, and so on. And there is plenty of evidence that institutions work hard to overcome such disadvantages. By changing the way institutions treat grades for admission purposes one source of unfairness should be alleviated. Others will remain, but come under much greater scrutiny because the myth of examination grade fairness will have been scotched. If Ofqual refuse to change their grading practices, universities must change their admission practices. Things can only get better.
If you enjoyed this article, you can also read Rob Cuthbert’s previous HEPI blogs: How to make HE admissions fairer, quicker and better and A-levels 2020: what students and parents need to know.
HEPI recently produced a presentation aimed at Year 12 and Year 13 students considering their options. See here.