This blog was contributed by Dennis Sherwood who has previously written about A levels.
At the Education Select Committee hearing on 2 September, Ofqual’s Chief Regulator, Dame Glenys Stacey, acknowledged that exam grades are ‘reliable to one grade either way’.
Two weeks later, at the hearing on 16 September, Gavin Williamson, the Secretary of State for Education, stated that ‘What is clear is that the exam system is the fairest and best system available to us’, a sentiment echoed by Nick Gibb, the Minister for School Standards, when he informed the Committee on 20 October that he believed exams to be ‘the fairest way we have of assessing children’s work, ability and aptitude’.
I am puzzled. To me, these statements are mutually contradictory. If exams are ‘reliable to one grade either way’, does that mean that A level results designated ABB are in fact any set of grades from A*AA to BCC? But if entrance to a university programme requires grades AAB, then any student with a certificate of ABB could well be rejected. How can this be ‘fair’?
Some further light on this has been thrown by the letter from the Committee’s Chair, Robert Halfon, to Gavin Williamson, dated 10 November, tabling some of the conclusions from the recent hearings.
The letter is hard-hitting, for example:
In early September Ministers made assurances to the House and us that all the relevant information and papers would be provided to this Committee, so that we could look into this matter. As yet, the delivery of this information has not materialised, and so we would ask that we receive the papers requested by our clerks by Monday 23 November at the latest.
It was revealing to us that they recognised this problem but simply accepted that they could not find a solution to it and chose to carry on. Ofqual conclude that what they had been asked to do was in fact an impossible task. Yet when the Chief Regulator came before us in June, she did not use the opportunity to raise any alarm bells with us, choosing not to do so publicly at all.
We regret that Ofqual decided not to raise wider concerns about the fairness of the model they were being asked to implement. They had every opportunity to do so when they came before us in June. Instead, they simply followed the ministerial direction and hoped for the best.
This whole episode calls into question Ofqual’s independence, its way of operating and its Board’s duty and ability for maintaining standards.
Each of those merits a blog in its own right, but here I wish to focus on ‘fairness’ and ‘reliability’, in relation to which this extract from Mr Halfon’s letter is most informative:
Ofqual was established under the Apprenticeship, Skills, Children and Learning Act 2009 to preserve the integrity of exam qualifications but it has no specific statutory remit to ensure fairness.
That may come as a surprise, for you might imagine that ensuring fairness to each individual student would be at the very heart of the examination system.
It isn’t. Mr Halfon is correct, for Ofqual is indeed constituted under Section 128 of the Apprenticeship, Skills, Children and Learning Act 2009, as amended by Section 22 of the Education Act 2011, which reads:
The qualifications standards objective
In section 128 of ASCLA 2009 (Ofqual’s objectives) for subsection (2) (the qualifications standards objective) substitute—
(2) The qualifications standards objective is to secure that—
(a) regulated qualifications give a reliable indication of knowledge, skills and understanding, and
(b) regulated qualifications indicate —
(i) a consistent level of attainment (including over time) between comparable regulated qualifications, and
(ii) a consistent level of attainment (but not over time) between regulated qualifications and comparable qualifications (including those awarded outside the United Kingdom) which are not qualifications to which this Part applies.
The obligation to ensure ‘a consistent level of attainment over time’ underpins the ‘no grade inflation’ policy, and the ‘consistent level of attainment [over space]’ is intended to ensure equality of academic standards.
The concept of ‘fairness’, however, is not mentioned. The possibility that some students might be treated unfairly by the exam system, or by this summer’s algorithm, therefore appears, from a legal point-of-view, to be neither here nor there. Presumably, any appeal based on grounds of unfairness would be dismissed out-of-hand. It seems as if school exams don’t have to be fair.
Which, indeed, they are not – for a host of reasons, not least the unreliability of the outcomes, the grades indelibly printed on candidates’ certificates.
But is there a catch? Although Section 22 does not talk of ‘fairness’, it does place a duty on Ofqual to ‘secure… a reliable indication…’.
In this context, what is the significance of Dame Glenys Stacey’s statement that exam grades are ‘reliable to one grade either way’? Is this a ‘smoking gun’, conclusive self-confessed evidence of a breach of the Act? Or would a clever lawyer argue that Section 22 does not specify what degree of reliability should be secured, and that ‘reliable to one grade either way’ is sufficient, and fully compliant with Ofqual’s legal obligation?
I’m not a lawyer, so I don’t know.
But as a member of the public, I have a view.
To me, exam grades should be fair, and a key aspect of ‘fairness’ is reliability. And grades that are ‘reliable to one grade either way’ can never be fair.
This is especially relevant in the context of the recent recommendations from both UCAS and UUK that universities should make admissions offers only after the A level results have been announced, so as ‘to make the system fairer by eliminating the use of predicted grades, which are often unreliable’.
In making these recommendations, UCAS and UUK appear to be placing even more importance on exam results, endorsing and trusting their reliability. Such trust, however, is misplaced. On average, about 1 grade in every 4 is wrong, with significant variations by subject (about 1 grade in 25 is wrong in Maths, nearly 1 in every 2 in History), and also by mark within subject (in essence, a script marked at or close to any grade boundary, in any subject, has a probability of about 50% of being awarded the right grade, so tossing a coin would be as fair).
Making offers after the results have been announced, when those results are so unreliable, doesn’t seem to me to achieve much. So perhaps UCAS and UUK should be demanding that Ofqual deliver grades that are not ‘reliable to one grade either way’ but ‘reliable, full stop’.
One defence, of course, is for Ofqual to claim that there is no credible alternative. This, however, is not the case, for it is very easy to award assessments that approach 100% reliability. But to do so requires a change in the policy by which the assessment shown on a certificate is determined from the corresponding exam’s marks, and also a change in the policy for appeals.
The pity is that Ofqual appear to be very reluctant to contemplate doing this.
Perhaps the current turmoil concerning the plans (or otherwise) for exams in England in summer 2021, along with Robert Halfon’s letter and the resulting fall-out, as well as the reactions to the recent proposals from UCAS and UUK, will increase the pressure on Ofqual for beneficial reform.
I do hope so.