Since its inception in 2017, the Teaching Excellence Framework (TEF) has become an integral part of assessing and recognising the quality of undergraduate teaching and learning in the English higher education system. Institutions in Scotland, Wales and Northern Ireland can participate on a voluntary basis. Managed by the Office for Students (OfS), TEF aims to evaluate providers based on teaching quality, learning environment, and educational outcomes.
In July 2022, the OfS introduced a revised TEF framework, incorporating a focus on ‘educational gain’, defined as ‘the distance travelled’ by students by McGrath et al. Astonishingly, the OfS did not provide a definition or framework for measuring educational gain but instead invited institutions to ‘demonstrate a clear articulation of their ambitions for educational gain, credible approaches for delivering this, and where possible evidence that it is delivered in practice’. This lack of definition and ‘anything goes’ approach made educational gain ‘the dark matter “holy grail” of higher education – we can believe it is out there but will never understand it’.
A joint webinar by evasys and HEPI in June 2023 delved into how institutions evidenced educational gain in their 2023 TEF submissions. While not exhaustive, the examples included here shed light on various institutional approaches.
Institutions often use graduate attributes to evidence educational gain. The University of Portsmouth and Queen Mary University of London, for example, have defined sets of attributes encompassing skills, competencies, and attitudes developed throughout the learner journey. These attributes go beyond the curriculum and form a central narrative in university policies. Staffordshire University utilises an Employability Framework, fostering personalised career support and interventions based on educational gain diagnostics.
Transition to University
Recognising the challenges students face during the transition to university, institutions implement various programmes. Staffordshire University’s FE2HE programme builds students’ professional identity, while the University of Portsmouth’s ‘Countdown and Connect’ programme utilises a gamified approach to engage students pre-arrival. The Being, Belonging, Becoming (BBB) framework at Portsmouth ensures inclusivity, with Welcome Ambassadors and buddy schemes fostering a sense of belonging.
Scholarship of Learning and Teaching
Queen Mary’s Academy Fellows and Staffordshire’s Academy Fellows programme exemplify institutions buying out staff time for scholarly projects. Queen Mary’s peer-led team learning project involves students facilitating groups in the year below, resulting in increased student confidence and self-efficacy.
Developing Employability Skills
Institutions emphasise opportunities for students to develop skills within and beyond the curriculum. Staffordshire’s EDGE enrichment programme, Portsmouth’s Seven Steps to Success programme, and Queen Mary’s Active Curriculum for Excellence integrate employability options, live briefs, and real-life projects into the learning experience.
Partnerships with students are crucial in designing learning experiences. Queen Mary’s Learner Intern programme and SEED Award, along with Staffordshire’s Student Futures Manifesto project, highlight collaboration with students to enhance co-creation, ensuring their voices are heard and valued.
Institutions face challenges in providing compelling evidence of impact, in other words, a pre- and post-assessment of change, learning, and development that can definitively confirm educational gain. Queen Mary adopts a holistic approach involving students, academics, and professional services staff. Staffordshire articulates individual and collective educational gains through strategic metrics, seeking to bridge the gap between the Access and Participation Plan and the TEF.
Learning is individual and unique
Oscar Minto, Reading University Students Union Education Officer for 2022-23, argues that educational gain is fundamentally unique and personal to each student. Students often take away more than we think and more than they may be aware. If ‘students themselves do not understand the gains they have made, capturing them as a metric is meaningless’. Minto stresses the need to guide students to think holistically about their learning journeys and educational gain beyond the classroom, recognising self-efficacy in society or representative roles, gaining skills through extracurricular activity, navigating resilience through setbacks, and achieving satisfaction and self-understanding.
This is a good point, as immediate change isn’t always observable in developmental programmes when impact can take years to become visible. We should not beat ourselves up, or indeed be beaten up, that educational gain is relatively invisible. Perhaps the bigger question is whether higher education institutions should be reprimanded or punished by government agencies in the absence of verifiable or measurable change.
Challenges and Reflections
TEF outcomes were released on 28 September 2023 for the majority of institutions. At the time, one-fifth of provider ratings were still ‘pending’ as the TEF panel finalised them. Whilst the parameters and mechanics for the next TEF will not be decided until closer the audit date, work must be done to evaluate the impact achieved by TEF and understand whether educational gain is a credible metric or can be considered a reliable and valid measure. For instance, it will be extremely useful to have an updated analysis of the 2023 TEF provider submissions along the lines of Diana Beech’s 2017 publication, Going for Gold: Lessons from the TEF Provider Submissions, where she undertook a detailed analysis of about one-third of all provider submissions, aiming to crack the code for success and draw lessons from the process.
In the meantime, here are some key considerations the sector may wish to address when evidencing educational gain for TEF:
- think outside the box as institutions strengthen their evidence-gathering practices and foster a culture of continuous improvement;
- focus on the entire student lifecycle, from pre-arrival to post-graduation, and the relationship between student experience and student outcomes;
- highlight what works and what needs improving;
- work in partnership with students’ unions from the outset to define a data and impact strategy to co-create shared understanding and avoid duplication and survey fatigue;
- build in evaluation from the start or as early on as possible; and
- remember that context is everything as institutions can only ever respond from the perspective of their own unique settings, goals, and challenges.
Love it or loathe it, like the National Student Survey, the TEF looks like it is here to stay for the foreseeable future. And, dare I say that it is possible the TEF can evolve into an effective instrument for uncovering the mystery of the dark matter of educational gain. But to do so will require more detail and credible metrics for recognising and enhancing teaching quality in the UK if, and it is a very big if, the OfS does not make an already complex and nuanced endeavour even more so, continues to collaborate with the sector and maintains a flexible approach with institutions.