Are the right freshers in the right places?

This blog was written by Dennis Sherwood, author of Missing the Mark: Why so many school exam grades are wrong, and how to get results we can trust, published by Canbury Press. It was recently reviewed in a HEPI blog by Rob Cuthbert, Emeritus Professor of Higher Education Management at the University of the West of England and Managing Partner of the Practical Academics consultancy. 

As the 2022 freshers arrive and start finding their way around, some will be supremely confident; others somewhat apprehensive; all will be excited. But will the right students be settling in to their new digs and dorms in the right places?

Here are some numbers you won’t find in the official UCAS, Department for Education, or Ofqual statistics.

For every 100 students awarded AAA for English A Levels in Chemistry, Biology and Physics – a subject combination well-suited to Medicine, for example – about 15 would have been given at least one grade B had their scripts been marked by a senior examiner, whose grade, according to Ofqual’s preferred terminology, is ‘definitive’ or ‘true’. If your institution used AAA in these subjects as the determinant of which students to admit, and which to reject, then about 15 per cent of your intake could well be under-qualified, occupying places that perhaps should have been taken up by students who received certificates bearing at least one B, when they should have received AAA.

For students taking humanities, the numbers are more startling. For every 100 students awarded AAA for Geography, Sociology and Economics, about 40 merited at least one B; for every 100 taking English Language, English Literature and History, about 50.

The fact is that A level grades in England – and AS and GCSE grades too – are, to use Ofqual’s own words, ‘reliable to one grade either way’. As a result, many of the candidates who missed an AAA offer for want of a single grade did so not because they were unlucky in revising the wrong topics, not because their teachers weren’t up to the job, but because Ofqual interprets its statutory duty ‘to secure that regulated qualifications give a reliable indication of knowledge, skills and understanding’ in the belief that grades ‘reliable to one grade either way’ are reliable enough.

Two other numbers:

• For every 10 students in England who sat eight GCSE subjects, only about one received a certificate on which all eight grades were ‘definitive’ or ‘true’ (to me, a much simpler word is ‘right’).
• Around 25,000 students in England who sat GCSE English and who were awarded ‘fail’ grade 3, and so consigned to the social dustbin of the ‘forgotten third’, would have been awarded ‘pass’ grades of either 4 or (for a few) 5, had their scripts been marked by a senior examiner.

These numbers, by the way, are not my invention. They are all derived from the measures of the reliability of grades as published by Ofqual in November 2018. Yes, you do need a ruler to determine what those reliabilities actually are, for Ofqual publish only a chart (figure 12, on page 21). And you do need a calculator or spreadsheet to combine the probabilities associated with different subject combinations. The raw data, however, was published by Ofqual nearly four years ago.

It may be that you have read this far and are thinking, ‘So what? None of this matters – as long as all our places are filled, that’s OK.’ Well, I understand that – sort of.

Alternatively, you may be thinking, ‘Really, is all this true? That’s outrageous!’, only to return to your emails, prepare that first lecture to welcome the new class, or go back to your research. I understand that too.

Let me point out, though, that higher education institutions – apart from the students themselves, of course – are the largest and most important ‘consumers’ of exam grades. In principle, higher education has enormous clout. But only if the institutions act collectively.

Ofqual has known about this problem since the mid-2010s and has been progressively forced into admitting just how unreliable exam grades are. So, for example, following an article in the Sunday Times on 11 August 2019, a ‘news story’ was posted on the Ofqual website admitting that ‘more than one grade could well be a legitimate reflection of a student’s performance’. Tell that to the student who missed an offer for want of that single grade.

Yet despite this knowledge, Ofqual has failed to fix the problem. And one reason for this is that insufficient pressure has been brought to bear. 

Higher education can exert that pressure.

But only if you believe that it matters that, on average,  about 1 exam grade in 4 is ‘non-definitive’ or ‘untrue’ (I prefer the simple word ‘wrong’). And only if you care enough to do something about it. Like discussing this with your colleagues in your faculties and departments. Like convincing your Principal or VC that this must be taken up at the highest level.

The power is in your hands. 

But only if you wish to use it.