It felt strange to be in the Houses of Parliament on Thursday 21st of March and not discuss Brexit. One of our panellists, Gervas Huxley, had been told by a student that our event was like discussing what colour to paint the bathroom while the building was on fire. Parliament on fire may be an apt metaphor for Brexit but the quality of students’ learning is a more important issue than the colour of a bathroom – hopefully at least to the readers of the HEPI blog.
We heard first from Helen O’Sullivan, Pro Vice-Chancellor of Education at Keele University, who summarised the research and evidence on how to teach in universities and argued we don’t have much evidence that this has had a big impact on teaching practice. She said that teaching in Medicine provided a notable exception, because of a reliance on rigorous evidence to inform teaching practice and the leadership of organisations like the General Medical Council in conducting trials into innovative teaching practices.
Helen argued that the Office for Students’ continuation of HEFCE’s work on learning gain did not seem hugely promising, and that she wouldn’t go over her criticisms of the Teaching Excellence Framework (TEF) as the Royal Statistical Society had said all she felt was needed on that. This, she argued, leaves a need for someone to collect evidence on university teaching practices and find ways to apply them to improving teaching. She stressed the need for an evidence base for cost-effective high-quality teaching as higher education moves onto ‘bumpy ground’ and faces financial constraints. Generalisable evidence was also needed, she noted, that would apply to the wide range of course styles across higher education.
This theme, that the very diversity of universities can be perceived as preventing effective evidence on teaching quality, was confirmed by Dr Ben Styles of the National Foundation for Educational Research (NFER). He told us about the process by which aspects of teaching in schools are evaluated through randomised controlled trials, how results are collated into meta-analyses and how these are summarised in a toolkit of the most and least effective teaching techniques.
This model has worked in schools thanks to the existence of standardised tests. The absence of standardised tests in higher education provides a major barrier to creating a generalisable evidence base. To this end, he recommended that universities should have some common questions in each subject’s assessments, so that students taking the same subject at different universities would have a core set of standardised questions on which different institutions could be evaluated.
Our final speaker, Gervas Huxley, a Teaching Fellow at the University of Bristol, picked up on this point. He argued that standardised tests are essential to improve teaching in universities. He said that schools had seen improvements in the last 20 years because of the pressure created by standardised tests, while universities had experienced a great deal of discussion, evaluation and bureaucracy around teaching but had not made the same improvements. He said the interests of the university sector had effectively blocked the introduction of standardised testing, citing the proposal by the OECD to do PISA-style testing at university ages which was opposed by the US and UK. However, he suggested, standardised testing was the best way to ensure improvement in university teaching because it would expose the sorts of issues raised in Academically Adrift, which exposed challenges in US universities because of standardised testing. Ultimately, he claimed, universities and university staff are incentivised to focus on research because it brings greater rewards on a personal and institutional level.
Some of the audience expressed doubts about the proposal for standardised testing, especially about the feasibility of producing tests for the diversity of courses or the desirability of teaching towards a single standard. However, it was suggested that for some subjects, such as Economics, there is a common core of subject matter that all undergraduate students could be expected to cover, which could therefore be assessed systematically. One audience member argued that moving towards standardised tests would threaten university autonomy, which is widely regarded as a necessary condition for a highly successful university system. A suggested counterpoint to this was Oxbridge colleges, who provide separate teaching for university-wide exams. It was suggested that this framework provides a guarantee that teaching would continue to receive investment in individual Oxbridge colleges.
Overall, a common theme from the discussion did highlight the need for a means of evaluating teaching quality at university level. One participant said of the TEF that it involved a great deal of data collection for very little differentiation. Another noted that the way the TEF assumes that all providers are high quality cannot be realistic and limited the effect that it could have to incentivise universities.
If higher education cannot (or will not) have standardised tests as a driver of teaching quality, then what will drive teaching quality?
Thank you Hugo. I’ve often wondered about standardised tests in higher education.
Lessons could be learnt from professional qualifications e.g Accountancy (ACA, ACCA, CIMA). Upon completing a degree in business, I studied CIMA for 2 years and took standardised tests. I had options of which education provider to study with – Kaplan or BPP. In this case, teaching quality is clearly measurable by student success rates.
Could the solution be a blend of standardised and institution-specific testing? The former could build towards a professional qualification.
Such professional qualifications may help students to demonstrate their intellect. Particularly if the university they graduate from does not have a strong reputation.
I hope that’s helpful.
Are we trying to “…..Evaluate Learning” or are we trying to evaluate teaching? Are the 2 the same?
I think not.
What exactly are we trying to measure and why?