Last week saw Gavin Williamson announce the closure of schools and colleges, along with the cancellation of primary school tests (SATS), GCSEs and A levels. Higher education institutions have followed suit, with all universities now suspending face-to-face teaching. The Secretary of State’s speech also included the cancellation of school performance tables for this year, presumably due to a combination of difficulty getting the necessary data and a sense of fairness to schools which are doing their best in these unprecedented times. I believe there may also be a case for halting university league tables temporarily.
I am generally wary of making recommendations for universities based on initiatives happening at earlier stages in the education cycle. University is not ‘big school’ and universities are autonomous organisations. However, schools and universities are facing many of the same challenges in these extraordinary times. How do they continue teaching? Do they have the right technological solutions for online learning? Do they continue to put in place some form of assessment and, if so, how? How do they ensure they are supporting the most disadvantaged students, who may not have access to the facilities they require at home or a safe space to work in?
Unlike school performance tables which are published by the Government, university league tables are published by third-party sources. The only exception to this is the Teaching Excellence Framework (not a league table but a method of rating universities), which is currently under review. The most prominent UK league tables are The Complete University Guide, The Guardian and The Times/Sunday Times’ Good University Guide. A full explainer of the league tables and metrics used can be found in HEPI report 101 – A Guide to UK League Tables in Higher Education.
While all the tables are slightly different, they use similar criteria. The tables include:
- Entry standards – the average UCAS tariff of students starting at that institution
- Data from the National Student Survey, including student satisfaction and teaching quality
- Research quality, as assessed by the Research Excellence Framework (there have been calls to cancel the next iteration, which will be conducted in 2021)
- Graduate prospects – the proportion of graduates in professional level employment or in graduate-level study
- The proportion of students achieving a first/2:1
- Completion rates
- Student-staff ratios
- Services/facilities spending
- Spend per student
- Value added – where students’ degree results are compared with entry qualifications
These league tables are published between April and September and are intended, at least in part, to help inform entrants the following year – so those published this year would be referred to as the 2021 league tables.
Like school performance tables, there are going to be significant challenges with the data used in the 2021 league tables. The National Student Survey runs between January and the end of April, meaning the results will be a mix of those who completed before the pandemic and those completing now, reflecting two very different types of student experience. It is also likely to suffer from lower response rates, given how much of the promotion of the survey is done on campuses and many students have now left their campuses.
Before the impact of COVID-19, league tables were already facing the challenge of limited data on graduate prospects. Due to the one-year gap in data during the transition between the Destination of Leavers from Higher Education (DLHE) survey to the new Graduate Outcomes survey, the 2020 table used the same data for two years (the 2016-17 DLHE data). For its first year, the Graduate Outcomes data will be published as experimental statistics, raising questions about whether it is appropriate to include within league tables. If not, they will be reliant on out-of-date DLHE data. This, combined with the NSS issues, means at least two of the input sources which play a significant impact in league tables may not be of a suitable quality to include.
It is not just this year’s league tables that will be affected. Some of the impacts on the data are going to be longer term. The 2022 tables will include the ‘entry standards’ for students who enter university this year. The cancellation of A levels has led to uncertainty about exactly what will be used in place of those awards, but last week’s Government announcement has indicated that it will be a combination of predicted grades and previous attainment. This is a form of contextual admissions, meaning universities are likely to be less focused on the UCAS tariff awarded to each student. This will make the entry standards data incomparable to previous years.
In terms of the graduate prospects data, for 2022 this will rely on the surveying of graduates who are entering the labour market this summer. Given the economic implications of Britain responding to COVID-19, the data are also unlikely to be comparable to previous years. Therefore, as for the 2021 league tables, the 2022 league tables will suffer from quality issues in at least two of its input sources.
As well as these challenges with the data quality, it seems to me there is also question of fairness. Given the circumstances we find ourselves in, it seems it is not the time for comparing universities against each other, but collaboration, as Anthony Seldon has called for. It is not down to universities or Government to take the call on whether these league tables should be published, as it is the decision of the third parties who run them. However, it seems to me that there is a fairly strong case for at least pausing their production for the 2021 and 2022 series.