WEEKEND READING: This year’s school grades – Ofqual’s consultation

This blog was kindly contributed by Dennis Sherwood, an independent consultant. Dennis has produced much of HEPI’s past output about A-Levels, including 1 school grade in 4 is wrong. Does this matter? and Trusting teachers is the best way to deliver this years’ exam results – and those in future years?

On Wednesday 15th April, Ofqual published their consultation for this year’s process for awarding GCSE, AS and A level grades, alongside a literature review on equality. My view is that the consultation document is thoughtful, comprehensive, and wise.

The document runs to 67 pages, and asks 37 consultation questions – far too much to discuss here. So let me focus on just a few key themes.

Overall, the process is:

1. For each subject cohort, exam centres (largely schools) are asked to submit, to the appropriate exam board, their ‘centre assessed grades’ and the corresponding rank order for all candidates.
2. The exam board will then apply a process of ‘statistical standardisation’ to check that the centre’s submission is in line with that centre’s appropriate historical results. The purpose of this is primarily to ensure that the overall results across the country are consistent with previous years and that there is no grade inflation.
3. If the centre’s submission complies, the ‘centre assessed grades’ are confirmed…
4. …but if not, the exam board will, unilaterally, over-ride the ‘centre assessed grades’ and impose grade boundaries on the centre’s submitted rank order (which will remain unchanged) to ensure that the ‘no grade inflation’ requirement is met.
5. The resulting grades will then be awarded.

There is much more too – about particular categories of candidate, about appeals – but this is the core process.

Importantly, the exam boards can over-ride the ‘centre assessed grades’ unilaterally, without consultation. So if I submit A*s for all my candidates, I risk being told that the outcome is all Es and I will have no opportunity to explain that, in fact, this year’s art class are all Picassos-in-the-making.

At first sight, that might appear to be greatly unfair. And, in reality, there will be some genuine ‘special cases’ that will be disadvantaged. But suppose that there was an intermediate step in which exam boards went back to the centres to enquire as to why the submission is out-of-line…

…in which case, as a centre, I know that might happen. So I have an incentive to submit all A*s, and hope that I can argue it out. And if I don’t, I have nothing to lose, so why not? That would lead to a totally unmanageable volume of ‘dialogues’, most of which would be spurious. And if a ‘dialogue’ were to take place, what would it be? A sterile, if not adversarial, argument in which, at the end of the day, the exam board has the power. So I think Ofqual are being wise in ruling all that out. Centres will submit, the exam boards will do their statistical analysis. They will either confirm the centre’s submission or change it. That’s that. Yes, there will be some who lose out, but – in my view – that just happens to be an unfortunate, but necessary, price to pay. And let us not forget the huge, largely unknown, ‘price’ that has been paid over the last decade or so as a result of the unreliability of exam grades.

Given that the exam boards have the unilateral power to draw the grade boundaries on the centre’s rank order, a moment’s thought will verify that all a centre need submit is that rank order, and the exam board can then place the grade boundaries in the ‘right’ places as determined by their statistical models. Why, then, bother to submit the ‘centre assessed grades’ at all, especially since there is the possibility that those submissions might all be changed?

In my view, the ‘centre assessed grades’ are a ‘good idea’, because, perhaps surprisingly, they help the centres, for the key task for every teacher is to determine the candidate rank order. And that rank order must be such that every rung on the ladder is occupied by only a single candidate. There can be no ‘joint 15ths’, as only a ‘clean’ rank order enables the exam board to position grade boundaries wherever they might choose. So every teacher must judge that Sam is higher in the rank order than Chris, but below Pat. Doing that for the whole cohort is hard, really hard: inevitably there will be ‘clusters’ where it is genuinely difficult to distinguish between the candidates.

So suppose that a teacher determines an initial draft rank order, perhaps with some clusters, and then superposes grade boundaries that result in a grade distribution that complies sensibly with history. This will identify which candidates are relatively distant from any grade boundary (and therefore less at risk from an exam board-enforced boundary change) and which candidates are close to a grade boundary (and so are at greater risk). The teacher can then focus particular attention on the candidates close to the grade boundaries to get the rank order as right as possible. The grade boundary problem is more troublesome for GCSE, with 9 grade boundaries, than at A level with 6 – which might raise the question of “why so many grades?” – but we will all have to live with that for the time being.

Rank order rank order rank order. It’s all about rank order.

And in compiling the rank order, my belief is that it would be unwise for any one teacher to decide alone. Far better to work in small teams, perhaps to have around the table someone from a different subject, who can help review the factors being taken into consideration (bearing in mind Ofqual’s advice about ‘holistic judgements’). There might be an overall ‘supervisory board’ overseeing the whole process, possibly including one or more governors or even parent representatives (not, of course, parents of candidates) so as to help build trust within that oh-so-important community of parents and carers. Maybe neighbouring schools might mutually agree some form of ‘external examiner’ roles. And a truly challenging problem-to-solve is how to gain the trust of that even-more-important community: the students.

Ofqual’s consultation document also explores the possibility that centre submissions should be accompanied by a ‘Head of Centre declaration as to the accuracy and integrity of the information provided’. I think that is a really good idea. It encourages teamwork and accountability, and, potentially, has significant cultural implications too – the Head cannot ‘turn a blind eye’ to that lazy chemistry teacher or the belligerent historian.

Yes, as I’ve said before, it’s all about trust.