Next week includes one of the most important days in the UK higher education calendar for many years as we will see the results of REF2021, which stands for the Research Excellence Framework. It is a detailed measure of the strength of UK research, overseen by Research England for all four parts of the UK.
To hugely simplify a complicated process, the REF uses panels of experts in different subject areas to consider three characteristics of institutions conducting publicly funded research: research outputs; research impact; and the quality of the research environment.
- On Monday, institutions will receive their own REF results;
- on Tuesday, institutions will receive all the results;
- on Wednesday, the media will get their first look; and
- on Thursday, the embargo will lift, with all the information made available on the REF website.
So here are seven things to note while we are awaiting the outcomes.
- The REF results will lead to a flurry of unofficial league tables and a jockeying for position among institutions that undertake research, which it would be wise to take with a pinch of salt. I was relatively new to higher education policy back at the time of the 2008 RAE (the predecessor to the REF) and remember being amazed by the speed with which one of our two oldest universities refuted in detail a media article which claimed the other one had done better in that year’s Research Assessment Exercise. It is sometimes said that Britons love to turn diversity into hierarchy and the huge amount of data produced in the REF process make this task relatively easy, even if such behaviour is eschewed by Research England itself. So in short, we are likely to see the sector, yet again, express general disapproval of rankings while simultaneously working hard to ensure any rankings reflect their own institution as positively as possible. It reminds me of the time I walked past some banners at the Institute of Education that declared it to be the best place in the world for education while on my way to a seminar they were hosting on why rankings should not exist.
- Some people will be disappointed. I have asked pretty much every university I have had close contact with over the past year how they will do in the REF and all of them (except one) has told me with certainty that they expect to do significantly better at an institutional level in REF2021 than last time around at REF2014. It is possible that most will get a better assessment of their research than last time around – that’s progress – but they won’t all do so when the results are converted into Quality-related research funding in due course for the pot of money is not bottomless like Mary Poppins’s Magic Carpet Bag. Moreover, people on individual REF panels whisper that there may be areas where the REF shows we are perhaps not quite as good as we thought we were – identifying such issues is one very good reason for running the whole exercise. To take an obvious example given the public health crises of recent times, the results of Main Panel A, covering medicine, public health and biological sciences, will reveal if we have been investing enough to maintain our standing in such areas over the medium-to-long term relative to other countries.
- While there will be lots of complaints about the bureaucracy and costs involved in the REF (some of which are likely to be exaggerated), as Voltaire said about God, if the REF did not exist, it would be necessary to invent it. You may think I am making a point about university management, as REF results are widely used as a management tool, but I am thinking more of the battle for financial resources. In the fight for resources, you need to be able to show evidence of quality and value to funders, whether it is HM Treasury, charities or corporates – and these bodies also want to see those they fund being held accountable for past spending. I remain doubtful that we will hit the UK Government’s target of having 2.4% of GDP spent on research and development by 2027 but we would be even further off course without hard evidence to justify the policy.
- The reasons the REF survives, despite strong criticism from many of those it affects, are linked to the processes followed. The smooth and collaborative way the exercise is organised by Research England (and formerly Hefce) is well respected – people may dislike the principle and existence of the REF but the complaints are rarely aimed at those organising it. Just witness, for example, the warm response to the flexible way in which Research England tweaked the REF timetable to take COVID into account. It is a huge team effort of course, but it is also a tribute to the calm and experienced leadership of David Sweeney, Research England’s CEO, who is soon off to pastures new after 15 years leading REF issues at Hefce / Research England. He will be sorely missed. The other process point that protects the REF is the methodology: peer review. However imperfect, this remains the best way of evaluating the quality of academic research, as the regular debates about moving to a metrics-driven research assessment exercise show. To paraphrase Churchill on democracy, peer-review is the worst form of evaluation – except for all the others that have been tried. I am not arguing, by the way, that the REF is perfect: no one does. Not every piece of academic work will have been fairly assessed, given the large volumes involved, but the scale of the REF also means that many such issues get ironed out in the aggregation of the results.
- Each RAE/REF is different to the one before. This stops people from gaming the system too much and allows incremental improvements to be made. Last time around, the big change was the addition of impact. (As a general election candidate in a university city in 2010, I won one vote – though only one – by reminding a voter of a manifesto commitment to delay the impending REF ‘because of doubts about whether there is a robust and acceptable way of measuring the impact of all research.’ After the election, such a delay did occur, against the very firm wishes of civil servants, but impact survived and is now embedded and largely accepted as part of the evaluation process.) This time around, one big change on the back of the official Stern review is a hefty increase in the number of staff submitted as part of the process: unlike the past, all staff with significant responsibility for research are being included, as a way of boosting – in Stern’s words – ‘career choices, progression and morale.’ But the new rules still give latitude to institutions and there are are surprises in the number of people submitted at an institutional level, with for example, some institutions that have been better known for teaching than research submitting large numbers. Will we also see hidden gems of research, contrary to the general push towards concentration and selectivity? We already know the next REF will be different too because, in her time as Minister, Amanda Solloway told us as much at the most recent HEPI / Elsevier research conference and this is now the focus of the ‘Future Research Assessment Programme’ (or FRAP).
- This week sees the results of the REF but not the full consequences of the REF. How the results get converted into funding comes a little later and will be different in Wales and Scotland to England and Northern Ireland, given the way devolution works. This is another reason for taking those league tables on the back of this week’s results with a pinch of salt. How the money is distributed is no less controversial than the existence of the REF. This was hinted at by MillionPlus on Thursday, when their CEO, Rachel Hewitt, said: ‘MillionPlus has made repeated calls for the way research is funded to be made more equitable’.
- While the REF results will, rightly, be the focus for the next few days, they will only ever be one part of what makes up a healthy research base. Most research is underfunded relative to its true costs and, as university teaching income continues to be squeezed, there are other funding shortfalls beyond those related to research that need to be made up from surplus-creating activity (such as educating international students). On Thursday, the Russell Group warned, ‘the average deficit per UK undergraduate taught is set to more than double from £1,750 in 2021/22 to around £4,000 in 2024/25, with deficits across all subjects.’ There are lots of things beyond healthy finances that you need for a healthy research environment as well, such as a pipeline of research staff, good working conditions, sensible migration rules and so on, as outlined in the Government’s R&D People and Culture Strategy.
Over the next few days, HEPI will be running a series of blogs capturing a range of views on the REF from those who have experienced it first hand. To declare an interest, Research England is a HEPI Partner, and we are very grateful for the support we receive from them, but we will be covering a range of views, from the positive to the critical. We hope these will provide context and stimulate healthy debate about the right way of evaluating UK research at the start of the third decade of the twentieth century.
In the meantime, if you are waiting for the results, good luck!