- This blog was kindly authored for the HEPI 20th Anniversary Collection by Dr Helen Carasso, Honorary Norham Fellow in the Department of Education at the University of Oxford, and Andrew Plume, Vice President of Research Evaluation at Elsevier.
- In August, we are running chapters from the Anniversary Collection as a series of blogs. This piece is the second chapter from that collection.
To measure is to know
Attributed to William Thomson, 1st Baron Kelvin (1824–1907)
As HEPI marks 20 years of debate and constructive criticism on matters relating to higher education in the UK and beyond, it has invited expert researchers, practitioners and commentators to reflect on the way that tertiary education has developed over that time. To provide a context for these contributions, we first begin with summary data, which illustrate shifts in the scale and scope of UK higher education during the life of HEPI to date.
The large majority of the figures used here to create an outline of the size and shape of the sector (and increasingly sectors, with the devolution of responsibility for education policy within the UK during this period) are drawn from publicly available sources such as the Higher Education Statistics Agency (HESA), the admissions service UCAS and Funding Councils / Office for Students (OfS).1 This reflects not only the accountability that is expected of organisations that receive public funds, but also a long-standing collegial attitude among institutions toward data sharing and re-use.
The sector’s commitment to the collection and publication of a wide range of data about its activities, participants and resources can be traced back to the Robbins Report of 1963.2 The review committee decided to base its recommendations on evidence-based forecasts of likely demand for higher education up to 20 years ahead. These were prepared by a group of economists led by Claus Moser, who was then at the London School of Economics. The resulting data became one of the foundations for the findings of the committee and its report emphasised the importance of such an evidence base, recommending that sector-wide data-collection along the lines developed by Moser and his team should continue.
As anyone who works within a planning section, finance department, research services office or registry of a UK university today knows, annual data returns cover most aspects of an institution’s activities, inputs and outputs. So consistent (and hence comparable) is this data that, from around the time that HEPI was created, these figures formed the basis for annual Performance Indicators (PIs). PIs are devised and published by the Funding Councils / OfS, each accompanied by an institution-specific ‘sector-adjusted’ benchmark.3
The consistency in collection and reporting within data for a single year may though be weaker when attempting comparisons between different years. This is because definitions of what is to be recorded may have been refined, or sometimes adjusted more radically, in response to changes in the operation or regulation of the higher education system. In the information that follows, such variations across the 20-year period are noted, and their potential implications for the data and their interpretation are discussed.
There is also the underlying question of why such changes were made: are they simply a fine-tuning of previous practice; were they needed to capture the changing nature of provision and / or institutions; or do they highlight aspects of institutional behaviours that there is political will to influence? As such, the emphases that have been placed on data collection across the sector over time can offer signals of shifts in the sector itself, or in the way it is thought about by governments and other policymakers.
One such substantial change has been the move to consideration of aspects of higher education, most notably student recruitment, as operating within a marketplace. The potential for anti-competitive behaviours in such a marketplace has drawn UK higher education providers into the orbit of the Competition and Markets Authority.4 Sectorspecific guidance issued by the authority has resulted in caution among institutional leaders about the extent to which it is appropriate or permissible to continue to share data on activities such as student recruitment. While there is clearly a continued will among professional leads to collaborate within networks of shared-interest groups across the sector, it is possible that another 20 years from now we may find that at least some part of the evidence-base that Robbins advocated for so successfully is no longer readily accessible.
UK Higher Education
To maintain standards within the sector, and hence to protect student ‘consumers’ within this ‘market’, the use of the title ‘University’ and the power to award degrees (Degree-Awarding Powers, DAPs) are both strictly regulated. At the same time, market forces have encouraged a growth in providers, with many in England taking advantage of changes in legislation that enables smaller, often specialist, institutions to register with the OfS for university title and / or DAPs.
As the first shows, the significant increase in the membership of the higher education sector in England over the past 20 years has not been reflected elsewhere in the UK. Student number growth however has been more evenly distributed across the country.
As the unit-of-resource available to institutions to educate each home undergraduate has fluctuated in real terms over this period, universities have responded in different ways to try to make up any underfunding on teaching: for example, many have sought to increase the number of students on programmes with uncapped fees.7 These are mainly either fullfee paying international undergraduates (which since 2021 includes those from the EU) or taught postgraduates (from home and abroad). This has resulted in a shift in the balance of the income streams for the higher education sector in each of the four countries of the UK.
The financial sustainability of UK universities therefore relies on cross-subsidies from one area of activity to another.9 Such crosssubsidies are made apparent through TRAC (the Transparent Approach to Costing or TRAC), which was developed by the sector to enable its members to meet Funding Council expectations concerning the accounting of public money. TRAC was, however, only in its infancy 20 years ago and hence neither universities nor the public bodies responsible for their oversight then had a definitive picture of relationships between income streams and activities undertaken.10 TRAC is now used routinely and, as applied to the annual accounts of institutions, has led the Office for Students to highlight what it perceives as risks that some universities face because of overreliance on fees from international students (often from only a small number of countries) to cross-subsidise other activities.11
Teaching
The growth in home (UK-domiciled) undergraduate numbers, although accounted for in part by demographic changes, also represents a significant increase in participation rates for younger people, with the politically totemic 50% figure first passed in 2019.12 This growth reflects greater diversity in the prior experiences and educational qualifications of new students too.
This diversification has been encouraged by the growing emphasis (both from within institutions themselves and from politicians and regulators) on widening participation.
Understanding of what widening participation means in practice has evolved from consideration of straightforward measures of the educational and social background of an applicant to more nuanced indicators, such as comparison between the academic achievements of an individual and the average grades of those with similar profiles. England’s Director of Fair Access and Participation, John Blake, now also emphasises that universities within his remit should be acting to achieve equity not only in the admission but also in the progression and completion of students in light of this contextual data.13
To assist institutions to assess their relative achievements in equity within their undergraduate education provision, HESA introduced annual Performance Indicators in 2003. These monitored the relative admission, continuation and graduate destinations of those from low-participation neighbourhoods or state schools and of those with a disability. As the table shows in relation to state school entrants, despite a UK-wide increase, trends have not been consistent across the UK; increases in England have been offset by near flat or declining rates elsewhere.
When taking a UK-wide perspective, it is important to consider individual institutions in relation to their subject-mix and the average age and prior qualifications of entrants. To accompany the Performance Indicators, HESA therefore introduced ‘adjusted sector benchmarks’ that take these factors into account. Even with this institution-specific dimension, these national Performance Indicators were withdrawn in 2022 because the emphases of regulators in England, Scotland, Wales and Northern Ireland had gradually diverged to reflect more local policies and priorities. Each part of the UK now publishes its own figures.
Within institutions, there is a concern that measures such as the type of school most recently attended or the participation rate of home neighbourhoods, while straightforward to collect with a high level of accuracy, may not be a good proxy for the socioeconomic status of an applicant.15 It has also been suggested that well-informed families may make choices, in particular about where children should study for their Level 3 qualifications to ‘game’ the admissions process, in light of the widening participation expectations that are placed on institutions.16 Nevertheless the continued political and wider societal interest in who goes to higher education, and where and what they study, means that direct (and in some cases proxy) indicators of the social and educational backgrounds of applicants for first degrees are highly likely to continue to be collected and monitored.
This long-term interest in equity within education for undergraduates is now gradually expanding to cover graduate students pursuing both taught and research degrees. There is however little or no overarching policy directing universities to focus on specific indicators that they might use to monitor progress, and therefore no sector-wide collection of data on the entry profiles or comparative completion rates of graduate students.
Notwithstanding this particular ‘gap’ in the data that institutions are obliged to return to HESA, a substantial amount of data about the operation of members of the sector is collected and published annually. This can provide the basis for informed insights into aspects of higher education across the UK, but when considering teaching may also need to be viewed in the context of external factors such as demographic shifts or global economic downturns to appreciate the full picture.
Research
Successive periodic national research assessment exercises (known since 2014 as the Research Excellence Framework, which evolved from earlier approaches) and governmentcommissioned reports on the UK research base since 2011 have claimed that the UK ‘punches above its weight’ versus international comparators.17 With a relatively small workforce and modest Research and Development (R&D) budget, the knowledge created within the higher education sector does indeed have an outsized impact on the world stage in terms of output, influence and collaboration.
The table shows peer-reviewed publications including at least one UK-based author (most of which are based in higher education institutions) have more than doubled since 2002. Yet owing in large part to the rise of Chinese contributors to the English-language literature, the UK’s share of global knowledge production has in fact declined over this period.
To understand the influence that this research has on other researchers, we consider Field-Weighted Citation Impact (FWCI), an indicator that accounts for variations in the frequency and timing of referencing behaviours of different fields of research. This indicator stood at a very high level in 2022 relative to the global baseline (1.53 vs 1.00) and has increased since 2002 despite the underlying growth in the sheer volume of publications it represents. The sense of growth in quality over and above the growth in output is reinforced by the UK’s share of the top-cited 10% of world publications rising from 15.8% two decades ago to 17.6% today.
It is important to recognise that research is a team sport, and that publications often reflect the contributions of multiple researchers based in different countries and in organisations other than universities. During the last 20 years, the UK has become an increasingly connected player in the global research system. In 2002, less than 30% of research with at least one UK-based author included at least one other author from another country; by 2022, this figure has more than doubled to 63% and far outstripped the world baseline of 22%. In the same period, UK publications reflecting academic-industry coauthorship – a sign of collaboration and innovation potential – has also increased significantly and is more than twice the global rate.
Thus, despite broader political moves to the contrary, these data clearly demonstrate that the UK higher education sector remains a significant member of the international research community and continues to build influential links and collaborations.
Notes
- Nick Hillman, One for all or all four one? Does the UK still have a single higher education sector?, HEPI Report 129, April 2020 https://www. hepi.ac.uk/2020/04/16/one-for-all-or-all-four-one-does-the-uk-stillhave-a-single-higher-education-sector/
- Robbins committee, Report of the Committee on Higher Education, 1963 http://www.educationengland.org.uk/documents/robbins/ robbins1963.html
- https://www.hesa.ac.uk/data-and-analysis/performance-indicators
- Competition and Markets Authority, UK higher education providers – advice on consumer protection law: Helping you comply with your obligations, 2022 https://assets.publishing.service.gov.uk/ media/6475b2f95f7bb7000c7fa14a/Consumer_law_advice_for_ higher_education_providers_.pdf
- Office for Students, Register of universities in England https://www. officeforstudents.org.uk/advice-and-guidance/the-register/guideto-the-ofs-register/; Scottish Government, Register of universities in Scotland https://www.gov.scot/policies/universities/; Register of universities in Wales https://www.gov.uk/check-university-awarddegree/recognised-bodies-wales; Department for the Economy, Register of universities in Northern Ireland https://www.economy-ni. gov.uk/articles/higher-education-policy. These figures exclude the group of institutions known as ‘alternative providers’, of which a UK government survey in 2012 identified almost 700, although 114 were either closed or no longer offering higher education within two years – John Fielden and Robin Middlehurst, Alternative providers of higher education: issues for Policymakers, HEPI Report 90, January 2017 https://www.hepi.ac.uk/2017/01/05/nearlythree-quarters-alternative-providers-will-remain-unregulated-highereducation-research-bill-becomes-law/
- Source: HESA
- The unit of resource available to an English university to teach each home undergraduate in 2022/23 is around £9,300 in cash terms, on average. This represents a real-terms reduction of £1,700 over ten years in real terms, as the value of the cap on tuition fees has decreased by 18% – Institute for Fiscal Studies, Annual report on education spending in England 2022, https://ifs.org.uk/publications/ annual-report-education-spending-england-2022, 2022, p.10
- Source: HESA
- The pattern of these cross-subsidies is discussed in more detail in Nick Hillman, From T to R revisited: Cross-subsidies from teaching to research after Augar and the 2.4% R&D target, HEPI Report 127, March 2020 https://www.hepi.ac.uk/2020/03/09/from-t-to-r-revisitedcross-subsidies-from-teaching-to-research-after-augar-and-the-24-rd-target/; also see Universities UK, ‘Higher education in facts and figures: 2021’ at https://www.universitiesuk.ac.uk/what-we-do/policyand-research/publications/higher-education-facts-and-figures2021#finances
- https://www.trac.ac.uk/about
- Office for Students, Financial sustainability of higher education providers in England: 2022 update, 2022 https://www. officeforstudents.org.uk/publications/financial-sustainability-ofhigher-education-providers-in-england-2022-update/ and Office for Students, Financial sustainability of higher education providers in England: 2023 update, 2023 https://www.officeforstudents.org.uk/ publications/financial-sustainability-of-higher-education-providersin-england-2023-update/
- This target was first set by Tony Blair in 1999.
- John Blake, ‘No trade-off between access and quality on my watch’, Office for Students blog post, 10 February 2022 https://www.officeforstudents.org.uk/news-blog-and-events/blog/no-trade-offbetween-access-and-quality-on-my-watch/
- https://www.hesa.ac.uk/data-and-analysis/performance-indicators
- Colin McCaig and Neil Harrison, ‘An ecological fallacy in higher education policy: the use, overuse and misuse of ‘low participation neighbourhoods’, Journal of Further and Higher Education, vol. 39 no.6, pp.793-817 https://shura.shu.ac.uk/7805/1/McCaig_-_an_ecological_ fallacy_-_post_publication_version.pdf
- Connor Stringer, ’Private students could switch to state schools to get into Cambridge in bid to cheat the system, experts warn – as figures show those who move from fee-paying schools to grammar or sixthform education are a third more likely to get in’, Daily Mail, 17 May 2023 https://www.dailymail.co.uk/news/article-12091615/Privatestudents-switch-state-schools-Cambridge-experts-warn.html
- Laura Brassington (ed.), Research Evaluation: Past, present and future, HEPI Report 152, September 2022 https://www.hepi.ac.uk/wpcontent/uploads/2022/09/Research-Evaluation-Past-present-andfuture.pdf
- https://www.scival.com/landing