Thank you, OfS, for publishing Contextual admissions – promoting fairness and rethinking merit. The fundamental premise is that access to higher education is being unfairly denied to disadvantaged students, as evidenced, for example, by the fact that ‘in 2018, 18-year-olds from the most advantaged areas were 2.4 times are more likely to enter higher education and 5.7 times more likely to enter higher-tariff providers than those from the most disadvantaged areas’. The paper then advocates that contextual data, and not just grades, should be integral to admissions decisions.
This is already happening to an extent, for example, in targeting how best to widen participation and to inform decision-making on specific applications, as illustrated by some informative cameo case studies from the University of Bristol, York St John University, Oxford University’s Lady Margaret Hall, King’s College London and some other campuses around the world.
The overall message is that although contextual admissions are ‘not a panacea for fair access to higher education in England’, the current ‘implementation of contextual admissions does not go far enough’. The paper then lists six bullet points identifying action that the OfS intend to take to help implementation take place faster, including:
- ‘working to persuade league table providers to use measures that do not penalise contextual admissions’; and
- ‘exploring ways of using the OfS Challenge Fund to develop innovative approaches to diversifying routes into higher education, such as access and bridging courses.’
All good stuff.
But may I, OfS, suggest another action, please? Put pressure on Ofqual to ensure that public examination grades are reliable in the first place.
An assumption underpinning this paper (and much else too) is that the grades awarded to any student are reliable – that a B is a B, and not a C or an A. Under this assumption, an institution requiring A level (or equivalent) grades BBB for admission to a particular programme will, understandably, use this condition, in good faith, as a filter, and reject a candidate with grades BBC. A candidate with grades BBC will use this filter too, and so not apply. But suppose that the script awarded grade C had been re-marked by a senior examiner and up-graded to B. The outcome would have been very different.
In November 2018, Ofqual published data on the reliability of public examination grades: for example, the reliability of (all types of) maths, averaged over the entire cohorts at GCSE, AS and A level, is about 96%; geography, about 65%; English language, about 61%; history, about 56%. As an example of what those percentages mean in practice, consider A level history, for which 48,937 grades were awarded in summer 2018. If all the corresponding scripts were to be re-marked by a senior examiner, Ofqual’s reliability figure of 56% implies that about 27,500 grades would be confirmed, and about 21,500 would be changed. Those changes might be up or down, and possibly by more than one grade.
Here are the words used by Ofqual themselves:
The probability of receiving the definitive grade or adjacent grade is above 0.95 for all qualifications, with many at or very close to 1.0 (ie suggesting that 100% of candidates receive the definitive or adjacent grade in these qualifications).
My translation is that, in general, grades are reliable to only one grade either way. So perhaps the requirement that admission to a given programme is BBB should be re-written as ‘admission to this programme is BBB if your scripts were marked by a senior examiner, but CCC is OK if by anyone else’. Unfortunately, no one knows who marks any script…
Further Ofqual research digs deeper, moving beyond averages over whole subject cohorts, and across GCSE, AS and A level, to the reliability of the grade associated with a specific mark for a specific subject at a specific level. One result is that the grade associated with any script marked at or close to any grade boundary is about 50% reliable at best. Yes, tossing a coin might be more fair.
The implications of grade (un)reliability run deep, and add fuel to the fire of unfairness to disadvantaged students. Take, for example, appeals (or, to use Ofqual’s language, ‘challenges’ or ‘reviews’). In 2016, Ofqual changed the rules for appeals, intentionally making it harder to raise a challenge. But the fee remains. Even though the fee is refundable if the original grade is changed, the risk that the fee might be lost is a huge disincentive – a disincentive that falls primarily on the disadvantaged student in a disadvantaged school. My estimate is that over 90% of those who would be awarded an up-grade if they were allowed to appeal do not do so.
But the problem that really worries me is the student who is awarded an erroneously low GCSE grade, so setting up a self-fulfilling prophecy of failure. That low grade might deny the student access to the A level course. If the student does start the course, it might result in a too-low forecast of the A level grade, which could close any number of doors. But worst of all, it could damage the student’s self-confidence, and in the absence of external encouragement…
I haven’t done the research, but I wouldn’t be surprised if these problems are more prevalent, and more severe, among the disadvantaged. As a consequence, the already-disadvantaged become even more so.
One solution to all this is to take OfS’s nudge towards contextual admissions all the way: ignore grades altogether and use context, and the student’s commitment, alone. But if grades are to play a role, the least we can ask is that the grades are reliable. That a B is indeed a B, and would not be changed to a C or an A if a senior examiner were to re-mark the script. And yes, it is possible to award reliable assessments, even if each script is marked by an ‘ordinary’ examiner.