Counting what counts: a multi-dimensional approach to educational gain
This HEPI blog was kindly authored by Professor Billy Wong, Director of Research and Evaluation (Access & Participation) at the University of Reading. Billy has recently written the paper Rethinking educational gain in higher education: Beyond metrics to a multi-dimensional model, and blogs his thoughts on this below.
With the next iteration of the Teaching Excellence Framework (TEF) under redevelopment, and confirmation that it will look vastly different to TEF 2023, we have an opportunity to rethink the notion of educational gain – if it is to continue as a core assessment.
From learning gain to educational gain, the concept is appealing for its emphasis on understanding how students grow and develop over time, and the extent to which higher education institutions can make robust claims about their roles and contributions.
However, the Office for Students (OfS) left the definition and measurement of educational gain to individual providers to decide for themselves, which left the sector with a multitude of definitions. In the absence of a clear, shared definition of and approach to educational gain, the sector has tended to default to what is most easily measured.
Yet, an over reliance on student outcome metrics (such as the National Student Survey, continuation/completion or Graduate Outcome data) reduces the indicators of student development into just numbers. More concerningly, this approach meant student groups with small numbers may be lumped together or even excluded in various statistical analyses. When we focus on lived experience as headline statistics, the nuances are swept away.
Sector conversation
Recent sector work has explored the complexities of educational gain, from Fung’s (2024) analysis of Gold-rated TEF institutions to Quality Assurance Agency’s Collaborative Enhancement Project, which found diverse, developing but disparate approaches.
For individual institutions, a context-specific relevant approach makes sense, reflecting their own goals, priorities and practical considerations. But as a sector, including for the OfS, such freedom makes national comparison difficult if not impossible, and we revert to readily accessible and available outcome data.
Yet, educational gain must not only capture cognitive progress, but also the broader and holistic developments such as confidence and belonging.
The sector would benefit from a shared but flexible frame of reference for educational gain, which advocates for a diverse approach to evidence student growth over time.
A multi-dimensional approach to educational gain
Informed by the foundations of learning gain, this new paper proposes a multi-dimensional model of educational gain through three interrelated domains: cognitive and metacognitive, personal and affective, and social and cultural. Drawing on educational, psychological and sociological perspectives, these domains recognise the different aspects of student development, which also foregrounds the importance of longitudinal data from both qualitative and quantitative methods.
A multi-dimensional approach appreciates the student experience across the agency-structure spectrum. It provides an overarching frame of reference that enables institutions to tailor the specific approach as appropriate for their contexts. There will be differences across the sector in how institutions apply these in practice, but if the three domains (cognitive and metacognitive, personal and affective, and social and cultural) are broadly shared and operated as a thematic proxy across the sector, then we are at least in a position to explore how different institutions have collectively explored those dimensions.
For example, for cognitive and metacognitive development, it is conceivable that TASO’s Access and Success Questionnaire (ASQ) is adopted nationally to provide sector-wide comparable data with use value within and across institutions. In parallel, it is also conceivable to run a longitudinal qualitative study that unpacks how students articulate, reflect on and discuss their cognitive and metacognitive development.
Similarly, quantitative and qualitative methods can explore the extent to which students grow in confidence, resilience and self-efficacy, or whether they expand their social capital, sense of belonging or broader development as global citizens.
A multi-dimensional approach offers a unified lens for understanding educational gain that recognises sector benchmarks as well as local narratives. Without such a multi-dimensional view, the sector risks defaulting to established metrics that do not capture the full breadth of gains students achieve during their higher education.
What institutions can do
Short, funded pilot projects – supported by modest capacity-building grants – would give staff the space to test these methods before it is rolled out more widely. Contextually relevant reflective tasks could be strengthened and encouraged across programmes to encourage students to engage more critically with their own development. Crucially, it is important to ensure that any evidence gathered is conceptually robust and grounded in relevant theories of student progress and gains, for example: cognitive and metacognitive development, personal and affective growth, and social and cultural development. National-level benchmarks can be used effectively alongside the richness of context-specific data and evidence collected over time at the institutional level – reconciling national comparability with institutional distinctiveness.
What next?
If educational gain – and variations of it – is part of any next assessments, then the OfS should really be more explicit about what it expects from institutions. The ‘test’ from TEF 2023 to give providers the freedom to set their own criteria may be well-intended, but it served limited value for the sector, and presumably for the regulators themselves. A broad, flexible guiding principle or framework might provide the necessary coherence, preferably one that invites theoretical and methodological foundations in addition to the practical and pragmatic.
- To read more: Wong, B. (2025). Rethinking educational gain in higher education: Beyond metrics to a multi-dimensional model. Journal of Higher Education Policy and Management. https://doi.org/10.1080/1360080X.2025.2545624
Comments
Paul Wiltshire says:
One thing that it pretty obvious is the Educational cost – as we are encouraging half the population to get into £50k+ debt.
We need robust survey data to establish whether the career that graduates end up doing is in any way connected to the fact that they studied a particular course subject for three years. I think it is likely to show that most graduates end up in careers where there is no link to their degree or just a tenuous one. So how can we justify getting them into so much debt?
Reply
Replies
Paul Smith says:
That’s not what this blog post is about.
Personally, the thing I find most dismal in the notion of educational or learning gain is that of measuring learning. The best approach to this I’ve seen comes from a blog post by Paul Ashwin (https://www.researchcghe.org/blogs/2018-02-21-learning-gain-myths-and-possibilities/):
“At its best, focus on learning gain can be central to institutions’ strategies for teaching enhancement by providing rich evidence about what is working and what might be improved… This would involve examining the way that universities use learning gain evidence to enhance their teaching practices, rather than focusing on the measures of learning gain themselves.”
Reply
Rod Bristow says:
In learning, not everything that can be counted counts, nor can everything that counts be counted. But it is a big leap to then conclude that measuring learning doesn’t matter. Rigorous measurement is the route to fairness, the elimination of bias and evidence based teaching. The sort of experiences that lead to gains that cannot be measured are a good addition, but not a replacement for measured outcomes.
Reply
Ron Barnett says:
Professor Wong raises an important issue – that of educational gain – and in a nuanced way, not repudiating measurement but seeking a wide approach to it.
I support Professor Wong’s interest in seeking an approach to educational gain that reflects ‘how students grow and develop over time’, which accommodates the student’s advancements in ‘confidence and belonging’.
At the end of their undergraduate programme, not least at the end of the degree ceremony, students may be heard to say ‘Being here has changed my life’: how account for that self-reflection?
Professor Wong’s three-pronged approach, drawing on perspectives from education, psychology and sociology, is a major step forward – but (for my money) it doesn’t go far enough!
To those three perspectives, we surely have to add that of philosophy so as to address, head-on, the very concept of educational gain and its conceptual hinterland against the horizon of ‘higher education’. Issues include:
– the notion of education itself (as R S Peters reminded us 70 years ago, education is a value-laden concept and therefore a contested concept – so ‘educational gain’ must itself be a contested concept)
– the notion of higher education – does that term ‘higher’ signify particular forms of human development? (Compare ‘Bildung’ and Newman’s idea of liberal education.)
– yes, definitely, educational gain must take on ‘belonging’ but how understand that, and its relationships to ‘being’ and ‘becoming’?
– and what of the popularity being given today within the philosophy of education to ‘human flourishing’ and to ‘wellbeing’ – do those concepts hold water for ‘educational gain’?
– Professor Wong mentions (in the paper) Freire and students becoming critical of societal inequalities but what of the broader notion of criticality, let alone the narrower concept of critical thinking (once assumed and heralded as THE central concept of (Western) higher education?
– And now does not the idea of educational gain not have to doff its cap to matters of the Earth (ie to move from Wilhem von Humboldt to his brother Alexander and to acknowledge the call of the Earth (ie Latour etc). ie, in the mid-C21, one cannot say seriously say that one has gained much educationally as a student unless one comes to see oneself against an Earthly horizon?
– in brief, the idea of educational gain points us towards ontology as well as epistemology, to human ‘being’ and to what there is in the world as well as what it is we may come to understand about the world.
I sketch this out only to indicate that any serious effort to spell out the idea of educational gain just has to include the resources that philosophy can offer. So the disciplinary base has now at least to be that of philosophy, sociology, psychology and education (and in that order!). Of course, the hard-headed will say ‘and what of economics’ – a matter for further dispute …
Ron Barnett, UCL
Reply
Add comment