Excellent outputs demand excellent people, cultures and environments

• Research England have invited responses on their proposals relating to the People, Culture and Environment element of the Research Excellence Framework (REF), with a deadline of 1 December.
• Here, Stephen Curry (Professor of Structural Biology and Consul at Imperial College London and Director of Strategy at the Research on Research Institute) responds to the recent HEPI Policy Note on the REF – and outlines a very different view.

If you have only learned about the proposed changes to the Research Excellence Framework (REF) from Professor Sir Nigel Thrift’s recent HEPI Policy Note, then you are likely to be alarmed. According to Thrift, the proposal to reduce the weighting on research outputs in REF 2028 from 60% to 50%* to allow for an increase from 15% to 25% on the environment statement, newly expanded to consider People, Culture and Environment, means that:

• the world has turned ‘upside down’;
• outputs are ‘in danger of becoming a footnote’ in REF assessments, having been pushed ‘off the end of the table’;
• one of the main lines for defending government investment in research in UK universities has been ‘removed’; and
• instead of ruling the waves as a ‘science superpower’, the UK is now destined to sink beneath them. 

Worse still, in Thrift’s telling, UKRI and the HE funders in charge of the REF have suffered ‘an attack of the vapours’; they now prefer to stoke the warmth of collegiality through a greater focus on People, Culture and Environment (‘whatever that means’, says Thrift) and to cast aside any considerations of individual merit. 

Thrift’s is one of the more plangent critiques of the proposed REF reforms, but he leans too heavily on speculation, deploying the verb ‘seems’ no fewer than ten times. His cries for caution sound similar to those from Glasgow’s Vice-Chancellor Anton Muscatelli and Policy Exchange’s Iain Mansfield, but what their analyses also have in common is a lack of detailed engagement with the case for change. This has been laid out in the Harnessing the Metric Tide report (and its associated literature review) and in the report of the International Advisory Group which fed into the REF review process. 

Nor have these critics shown much awareness of reports and policy documents that have accumulated over the past 10 years from the Nuffield Foundation, the Wellcome Trust, the Royal Society, the Russell Group or indeed the UK Government, not to mention serious critiques of the rhetoric of excellence, all of which have highlighted how modes of assessment are locked in a tight embrace with research culture, licensing a hyper-competitive working environment that devalues research integrity, slows productivity, impoverishes problem solving, gives licence to biases that exclude talent and protects bullies and sexual predators who are judged to be research stars. 

Thrift might acknowledge of the systemic flaws in the research ecosystem but waves them away with the vague and unsupported claim that some of them ‘are being corrected in various ways.’ How exactly? Elsewhere he says efforts to tackle issues of EDI, bullying and harassment should not be used to ‘devalue the generation of ideas.’ I am sure it is in no way intended, but I suspect Thrift’s words will be read by junior researchers, particularly those from under-represented groups, as yet another signal that those in power are not serious about dealing with a culture that daily degrades their ability to do their best work. 

The REF proposals are emphatically not about sacrificing excellence on the altar of warm feelings. Rather they recognise squarely (as UKRI has done) that assessment shapes culture and, done badly, impairs the capacity of the system to generate the highest quality outputs. If we are truly interested in excellence that permeates the research system, we can no longer afford to take an optional-extra approach to research culture – the toll exacted by a ‘success at all costs’ mindset is too high. 

Thrift asks whether the REF is the right mechanism to address the systemic issues of modern-day research culture and that is a fair question. I would agree that while the REF cannot be expected to address all the ills currently afflicting research performance; it needs to be aligned with the efforts by universities, research institutions and funders to address broader questions of research culture. Our systems have to cohere. To achieve that we must confront the complex interactions between our conception of excellence and the problematic aspects of our working environment that it so evidently fosters. 

The current REF proposals mark an evolutionary step in that direction. It a step that is continuous with the development of the REF (to incorporate environment and impact components) since our nationwide Research Assessment Exercise was first conceived in the 1980s. What is being aimed at is a fuller richer conception of how to evaluate the research excellence in the UK. 

The proposals seek to arrive at a better balance of the various elements that denote real excellence. What is the right balance between those different elements? Nobody knows because nobody has complete information, and even the information we do have is given different weights by different people. We have a strong sense – even hinted at by Thrift – that the balance right now is not working. The REF 2028 proposals create the opportunity to have a sector-wide conversation, which has now started. It will continue with further consultations and work to delineate some indicators and descriptors of a positive research culture.

The consultation process is unlikely to arrive at a consensus – but that is OK. The system is so complex that it will not be possible to map out a route that can truly optimise the excellence of UK research. We need to be honest about that. What is important is that we enter into frank and informed discussions about how to arrive at a more complete understanding of the various aspects of research excellence that will best serve the UK. No one can say for sure right now where we will end up, though some interesting and constructive proposals have already been made. It is my hope that Research England and the other HE funders in charge of the REF will want to reward institutions for innovating the ways they evidence how their efforts to create environments in which their researchers can thrive enhances the quality and productivity of their research endeavours. 

The journey may well take time, but it also requires leadership. It is an opportunity for our universities, particularly the research intensives, to show that their warm words about people, culture and environment spring from a determination to realise a sophisticated, integrated and credible vision of how to achieve research excellence in all its dimensions.

*Explanatory note: Thrift suggests that outputs might be weighted as low as 40% but doesn’t explain how he arrives at this figure. The official documentation states that the assessment of outputs (Contribution to Knowledge and Understanding) will comprise 50% of the REF score, made up two components: panel assessments of submitted publications and a structured evidence-based statement that will provide a more contextualised explanation of the academic impact of the research performed by the unit of assessment. This statement will comprise at least one-tenth of the score for this component; i.e. at least 5% of the total score. The total score for the scholarly quality of the research output from the UoA is therefore 50%, not 40%.