Can we trust this year’s exam results?

Thanks to Dennis Sherwood for contributing this blog. Dennis has been writing about A levels on the HEPI from long before 2020. You can find Dennis on Twitter @noookophile.

On 18 March 2020, Boris Johnson announced the cancellation of the summer 2020 school exams. A few days later, on 21 March, HEPI published a blog entitled ‘Trusting teachers is the best way to deliver this year’s exam results – and those in future years?’.

That blog drew inspiration from Gavin Williamson’s statement of 20 March 2020, that:

The exam boards will be asking teachers, who know their students well, to submit their judgement about the grade that they believe the student would have received if exams had gone ahead.

Alas, my initial optimism was misguided. As we now know, teachers were not trusted but were second-guessed by the now-notorious algorithm, and their judgements ignored – until public outrage exploded.

This year’s exam-free process is different, with not only no algorithm, but also, apparently, no constraints to limit grade inflation. And ‘trusting teachers’ is explicitly the central theme: as Gavin Williamson said at the Downing Street press conference on 24 February, ‘We’re putting our trust firmly in the hands of teachers… there will be no algorithms whatsoever’.

This year’s grades will therefore be determined totally by teacher assessment, subject to external quality assurance by the exam boards that those assessments are ‘a reasonable exercise of academic judgement’. However, if anything goes wrong, it is the teachers – not the Government, not Ofqual, not the exam boards – who will take the flak. From the Government’s perspective, that could indeed be phrased as ‘we’re putting our trust firmly in the hands of teachers’, but to me, where the Government might put their trust misses the key point: whom the Government may or may not trust is ‘interesting’, but it’s whether the public – students, teachers, parents, admissions officers, employers – trust the grades that’s important.

Yes, it would be nice if those grades were to be widely trusted. But I fear that the Government, while declaring their undying and unbounded trust in teachers, have adopted policies that have put public trust in teachers in jeopardy.

‘The fairest way’

For months, every politician and senior official has been saying ‘exams are the fairest way of judging student performance’. Fairest way. Fairest way. Drumming the message time and again.

So here is today’s logic exam:

• ‘Exams are the fairest way.’
• Therefore any other way is at worst, unfair; at best, less fair.
• Teacher assessment is another way.
• Therefore teacher assessment is at worst unfair; at best, less fair.
• Especially when teachers have different points of view, apply different criteria and make different judgements.
• Especially when students have had different opportunities for learning and vastly different experiences over the last several months, as caused by the virus.
• Discuss.

Any student, any parent, who has heard that ‘exams are the fairest way’ must be puzzled that the u-turn to teacher assessment has in fact happened; disappointed that this year’s process has, officially, doubtful fairness; concerned that ‘another student’ might be treated differently, and therefore unfairly advantaged over ‘me’; worried that ‘I’ might be a victim. The ‘fairest way’ mantra has undermined public trust in teacher assessment, and I don’t see, or hear, any initiatives to rebuild that trust, other than the pitiful bleating of the Secretary of State.

But if the Government now tries to persuade us that ‘teacher assessments are the fairest way’, that will be interpreted as yet another u-turn. Given the Government’s track record for rotation, this would surely be brushed aside; a much more significant problem is that any admission that ‘teacher assessments are the fairest way’ makes it more difficult to re-instate exams as the ‘fairest way’ when exams will, at some time, return. The Government, presumably, does not wish to undermine exams, and so that problem is resolved by undermining teachers instead.

Oh dear. When will the penny drop that both exams and teacher assessment have an important, valuable and trusted role to play, with teacher assessment being fully trusted this year, and looking forward to a reformed system in the future in which both exams and teacher assessment each play a substantial and appropriate role?

This year’s appeals

Having undermined trust in teacher assessment from the outset, a safety net has been offered by confirming that ‘A student unhappy with their grade would submit an appeal to the school or college…’, with the possibility of  escalation to the exam board, who will confirm the teacher’s originally-awarded grade if they consider that it represents a ‘reasonable exercise of academic judgement’.

If public trust in teachers is low, there could be a very large number of appeals – especially since there are indications that appeals will be free, so appellants willing to take the risk of a downgrade have nothing to lose. Indeed, a surge in appeals appears to have been anticipated by the decision to publish the results earlier than has been the case in the past:

to allow more time for post results processes, including both appeals and transition activities for students moving between phases of education.

The pressure on the appeals process can of course be reduced if there are fewer appeals. Accordingly, on page 16 of Ofqual’s ‘Consultation Decisions’ document, we read:

To reduce the number of errors made and, in turn the volume of appeals, centres will be expected to tell their students the evidence on which their grades will be based, before the grades are submitted to exam boards. This will allow issues associated with, for example, absence, illness or reasonable adjustments to be identified and resolved before grades are submitted.

I find this statement troubling.

Will this be misinterpreted as saying ‘centres will be expected to tell their students their grades’, before those grades are submitted, let alone awarded? I know it doesn’t say that, but will people notice the subtle difference between the rather complex ‘tell their students about the process that will be used to determine their grades, and the nature of the evidence that will be considered’ and the much simpler ‘tell their students their grades’? – especially in the light of headlines such as ‘Let schools tell pupils their teacher assessment grades before submission, says exam board’.

Will teachers have to unwind this misunderstanding? When students, parents and carers come to them to say, ‘The rules are that you have to tell me my grades, now’, teachers will be faced with responding, ‘Well, that’s not quite what the rules really say…’ – a conversation which could well be difficult.

And when the teacher is through all that, the next demand, ‘OK. So how are you determining my grades?’, is not so easy either. Suppose the teacher replies, quite honestly, ‘We are taking into consideration [this coursework], [that test]…’. Inevitably, this will result in any number of ripostes from ‘but that’s not fair because I was not feeling well that day’ to ‘but my cousin’s school down the road is taking [whatever] into account too, so why aren’t you?’.

Avoiding confrontation

That’s all uncomfortable, so teachers, quite understandably, will seek to avoid such confrontations. So it’s in every school’s interests to define a local protocol, specifying in detail the elements of evidence to be considered, the weighting assigned to each and the rules by which the result is mapped onto a grade. The possibility, if not likelihood, that different schools will adopt different protocols is not any one school’s problem, for as long as a school complies with its own protocol in the same way for all students, the school should be ‘safe’. I hesitate to describe this as an ‘algorithm’, but that’s what I think it is; nor will I make (too much) of a meal about the fact that last year Ofqual directly refused to divulge the details of their algorithm, despite being told to do so by the Select Committee in their report of 11 July 2020, yet this year Ofqual are demanding that every school does exactly that.

Schools might also spot that the number of appeals they will have to deal with might be reduced by awarding grades likely to please. At A level, the allure of matching grades to UCAS predictions must be strong. UCAS predictions are already there, they have been taken into consideration in the appropriate admissions processes, and submitting them as this year’s actual grades pre-empts that most awkward question ‘why is the grade you awarded lower than the UCAS grade you predicted?’ And since UCAS guidance states that ‘predicted grades … should be aspirational but achievable’, that ‘but achievable’  should ensure that the school is safe from any challenge by an exam board as regards the school’s ‘reasonable exercise of academic judgement’.

For GCSE, there is no handy equivalent of UCAS predictions, so the temptation here is to bid up – a temptation fuelled by that rumour about Nearby High’s intention to do just that, let alone memories of last year, when schools that played by the perceived ‘rules’ lost out to those who didn’t.

If I were a teacher, I would defend my ‘reasonable academic judgement’ by reminding any external quality assurer that, in her evidence to the Select Committee Hearing on 2 September 2020, Ofqual’s then Acting Chief Regulator, Dame Glenys Stacey, acknowledged that exam grades are ‘reliable to one grade either way’: ‘So who are you to quibble over whether my student merits a 7 rather than the 8 that I judge to be fair?’ With, in reserve, ‘According to UCAS guidance, A level predicted grades should be aspirational but achievable. The school applied that principle to this year’s A levels, so, for fairness, the same principle must be applied to GCSEs too.’ Will an exam board, who quite possibly has an eye on next year’s exam income, really want to have a big fight with its customer? And do exam boards have the capacity to challenge 5 million GCSE grade submissions?

Back to trust

Overall, this year’s process has placed teachers directly, and alone, in the appeals firing line, so building-in gold-standard incentives for grade inflation – for A level grades to match UCAS predictions, for GCSE to be (at least) one grade up. That is likely to keep many people happy, and so avoid the nightmare of a multitude of long-disputed, acrimonious appeals. But that is not a universal ‘good’: the university admissions process will once again be thrown into turmoil and even the individual awarded those A*s might suffer real anguish when struggling to cope with a programme for which he or she is ill-qualified to pursue.

The ideal is for grades to be reliable and trustworthy, truly reflecting a student’s capabilities. And teachers are in a strong position to exercise the required judgement to do just that – they really do know their students best. But they must act with integrity and be trusted by students, parents, carers, admissions officers and employers to have done so.

So what are the Government, Ofqual and the exam boards actively doing to build that trust? What can teachers and schools do locally to convince students, parents and carers that they can indeed be trusted? What can organisations such as ASCL, NAHT, HMC, the Sixth Form Colleges Association, the Association of Colleges, NEU and the rest do to help?

If this year’s process turns out to be fraught, with ‘everyone’s a winner’ grades or embattled teachers overwhelmed by appeals, that will be bad. But perhaps not all bad, for it might fuel the fire for the fundamental reform – ensuring that exam grades are sensibly distributed as well as fully reliable and trustworthy, rebalancing the relative importance of exam results and holistic assessment in a student’s final award, scrapping GCSE, making the curriculum truly relevant and all the rest. Why, though, should a crisis be the driver of change? These changes should take place anyway and as a consequence of the success of this year’s process, not its failure. So the most important priority is that this year’s process works; that this year’s exam candidates are not only treated fairly, but trust they have been treated fairly too.