HEPI welcomes the recently published HEFCE proposals for the new Research Excellence Framework but calls for a re-think on the "impact" requirement

In a critique of the recently published HEFCE proposals for the Research Excellence Framework (REF) published today (15 October) by the Higher Education Policy Institute (HEPI), Bahram Bekhradnia, HEPI's Director, argues that these represent a considerable improvement on what was proposed previously but that there are a number of points about the proposal for impact that need to be re-thought - in particular that the academic impact of a piece of research will not be able to counter wider impact; and that the weight given to societal and economic impact should be the same in each subject. HEPI argues it seems unwise to attribute so great a weight to a feature that is in effect experimental- the HEFCE consultation paper itself acknowledges that the method for assessing wider impact has yet to be developed.

Commenting on the proposals published by HEFCE for the design and conduct of the REF, Bahram Bekhradnia said:

"The Government's original proposal was to replace the Research Assessment Exercises (RAE) with an assessment process based on measuring how much Research Council and other external income universities had earned. While the Research Assessment Exercise had been based on peer review - the assessment of the quality of research by experts in the disciplines concerned - peer review was entirely absent from the proposals: assessment was to be entirely quantitative. The process now proposed is radically different, and will recognizably be a development of the previous Research Assessment Exercises. The new proposals are a very great improvement on what was previously proposed, and the funding bodies are to be congratulated. Most important, as with the RAE, it is now proposed that assessment will be by peer review.

However, the major new factor is that the assessment of "impact" is now an important separate and explicit element - impact had not been mentioned in the earlier proposals. That has been added as an entirely new element that it is proposed should have major bearing on the overall quality assessment of a submission's quality. Panels are now required to assess ‘impact' separately, and the funding bodies' initial proposal is that 25 per cent of the final score that will be used for determining funding will be based on the impact score. And the proposals are clear that by impact is meant impact beyond the academic environment - specifically economic and social impact: impact is here defined explicitly to exclude academic impact. Nor is there any acknowledgement that the impact criterion may be more appropriate in some disciplines than others. Indeed, there is an explicit statement that it is expected that in all disciplines the impact factor will count for a similar amount.

This proposal needs to be handled extremely carefully. While it is understandable that those in government who provide funding for research wish to see some economic and societal benefit, it seems unduly limited not to be able to value outstanding research that, for example, may change the face of a particular, but perhaps narrow, aspect of an academic discipline as highly as other research whose academic impact may be less, but whose societal impact is greater. Despite the fundamental continuity with previous exercises, the extent of the change that the introduction of "impact" introduces is significant, and greater than any changes made previously. In the same way as the 1992 changes gave the message that it was not sufficient to produce good research if it was not communicated widely through publication, so the addition of "impact" as a requirement to achieve top grades will mean that producing and disseminating excellent research will not be sufficient if it cannot be shown to have had impact beyond the academic community. And in the same way as the 1992 changes fundamentally altered both the productivity and the publication behaviour of academic staff, so the proposed impact requirement will influence behaviour in ways that can only be speculated about at present."