HESA has completed its first round of the Graduate Outcomes survey, the new survey of graduates fifteen months post-graduation, which replaced the DLHE survey, which was only six months post-graduation. This is the first stage in running the new survey, which is the biggest annual social survey in the UK and is larger than any survey undertaken by the Office for National Statistics other than the Census. We should all be following its progression closely, as the data it produces is critical for government policy making, widely used to inform prospective students, including in the Teaching Excellence Framework and Unistats and provides key information about the graduate labour market which can contextualise the information provided by LEO.
Following the end of the first round of the survey, HESA has published an end of cohort review. This showed the first part of the survey received a response rate of 36%. Compared to DLHE’s annual response rates of 80% this may initially make some flinch. But I think, looking at the detail, there are four reasons why HESA can classify this as a positive outcome:
- It’s already been accepted that HESA isn’t going to get as higher response rates with Graduate Outcomes as they did with DLHE. When rethinking DLHE, it was agreed that six months out of university was too early a point to survey graduates. The fifteen-month point was agreed upon on the basis that the challenge of lower response rates would be offset by the new data being more useful, as it gives more meaningful information about graduate’s progression in the labour market. I believe HESA are aiming to get around 60% response rate for UK students.
- The students from each academic year are split into four cohorts, which make up one dataset. With this first cohort, HESA weren’t aiming for high response rates. Because this group has a large proportion of international students, who have lower response rate targets (largely due to the way the data is used, and the fact they tend to be more difficult to contact) the target response rate was set at 43%. So, at the end of this first cohort, HESA ended up only 7% away from their target.
- It was always going to be challenging to hit the targets in the first round of surveying. While HESA put a huge amount of work into preparations for Graduate Outcomes going live (I can say this confidently, given it was largely my job when I worked there) the scale of what HESA are doing this year shouldn’t be underestimated, contacting over 700,000 graduates. Taking on something of this magnitude was always going to bring up some teething problems.
- This first round only contains around 20% of graduates. By the time all four cohorts have been completed we will have a picture of graduates from the 2017/18 academic year. The biggest group of graduates in this cohort (around 70%) isn’t surveyed until September, so HESA still have time to continue to refine their processes and learn about how best to encourage graduates to engage. It’s too soon to be concerned about the overall response rate and the usability of the data, when the full dataset isn’t planned to be published until this time next year (however if HESA don’t tackle this by this point, we should absolutely be calling this out).
So, can this be classified as a good outcome? It’s largely too soon to tell, but early indicators are positive.