WEEKEND READING: Three reasons why the TEF will collapse under the weight of OfS  and DfE expectations

This blog was kindly authored by Paul Ashwin, Professor of Higher Education, Lancaster University.

The Office for Students (OfS) and the Department of Education (DfE) have big plans to make the TEF much more consequential. They want future TEF outcomes to determine whether institutions can increase their intake of students and their undergraduate tuition fees in line with inflation, which could mean the difference between survival or merger/closure for many institutions. These plans require that the OfS show that the TEF provides a credible measure of institutional educational quality, while also fulfilling the OfS’s central remit of acting in the interest of students. The OfS consultation on the future approach to quality regulation provides an opportunity to assess the OfS’s latest attempt at such a justification. To say it looks weak is a huge understatement. Rather, unless there is a radical rethink, these proposals will lead to the collapse of the TEF.

There are three reasons why this collapse would be inevitable.

First, the TEF provides a broad, if flawed, measure of institutional educational quality. This was fine when the main consequence of a TEF award was the presence or absence of a marketing opportunity for institutions. However, if the TEF has existential consequences for institutions, then a whole series of limitations are suddenly cast in a deeply unflattering spotlight. The most obvious of these is that the TEF uses programme level metrics to make judgements about institutional quality. It is both conceptual and methodological nonsense to attempt to scale-up judgements of quality from the programme to the institutional level in this way, as has been routinely stated in every serious review of the National Student Survey. This didn’t matter too much when the TEF was lacking in teeth, but if it has profound consequences, then why wouldn’t institutions consider legal challenges to this obvious misuse of metrics? This situation is only exacerbated by the OfS’s desire to extend the TEF to all institutions regardless of size. The starkest consequence of this foolhardy venture is that a small provider with insufficient student experience and outcomes data could end up being awarded TEF Gold (and the ability to increase student recruitment and tuition fees in line with inflation) on the basis of a positive student focus group and an institutional statement. How might larger institutions awarded a Bronze TEF react to such obvious unfairness? That the OfS has put itself in this position shows how little it understands the consequences of what it is proposing.

Secondly, in relation to the OfS acting in the student interest, things look even worse. As the TEF attempts to judge quality at an institutional level, it does not give any indication of the quality of the particular programme a student will directly experience. As the quality of degree programmes varies across all institutions, students on, for example, a very high quality psychology degree in an institution with TEF Bronze would pay lower tuition fees than students on a demonstrably much lower quality psychology degree in an institution that is awarded TEF Gold. How can this possibly be in the student interest? Things get even worse when we consider the consequences of TEF awards being based on data that will be between four and 10 years out of date by the time students graduate. For example, let’s imagine a student who was charged higher tuition fees based on a TEF Gold award, whose institution gets downgraded to a TEF Bronze in the next TEF. Given this lower award would be based on data from the time the student was actually studying at the institution, how, in the name of the student interest, would students not be eligible for a refund for the inflation-linked element of their tuition fee?

Thirdly, the more consequential that the TEF becomes, the more pressure is put on it as a method of quality assessment. This would have predictable and damaging effects. If TEF panels know that being awarded TEF Bronze could present an existential threat to institutions, then they are likely to be incredibly reluctant to make such an award. It is not clear how the OfS could prevent this without inappropriately and illegitimately intervening in the work of the expert panels.  Also, in the current state of financial crisis, institutional leaders are likely to feel forced to game the TEF. This would make the TEF even less of an effective measure of educational quality and much more of a measure of how effectively institutions can play the system. It is totally predictable that institutions with the greatest resources will be in by far the best position to finance the playing of such games.

The OfS and DfE seem determined to push ahead with this madness, a madness which incidentally goes completely against the widely lauded recommendations of the TEF Independent Review. Their response to the kinds of issues discussed here appears to be to deny any responsibility by asking, “What’s the alternative?” But there are much more obvious options than using a broad brush mechanism of institutional quality to determine whether an institution can recruit more students and raise its undergraduate tuition fees in line with inflation. For example, it would make more sense and be more transparent to all stakeholders, if these decisions were based on ‘being in good standing’ with the regulator based on a public set of required standards. This would also allow the OfS to take much swifter action against problematic providers than using a TEF-based assessment process. However things develop from here, one thing is certain: if the OfS and DfE cannot find a different way forward, then the TEF will soon collapse under the weight of expectations it cannot possibly meet.