This guest blog responds to HEPI’s recent survey on attitudes towards illegal drugs. It has kindly been contributed by Dr Keir Irwin-Rogers.
Keir is a lecturer at the Open University, a researcher on alternatives to imprisonment and the author of a forthcoming paper on ‘Drug prohibition, Inequality and Consumer Capitalism: A Toxic Trap.’ This will ‘highlight the fundamental flaws and harms generated by the UK’s prohibitionist drug policies in the context of high levels of socioeconomic inequality and a culture of consumer capitalism.’
HEPI’s role is to encourage debate. So we are pleased to provide a platform for this reinterpretation of our results. As always, publication of a guest blog does not signal an endorsement for all the conclusions – see Editor’s Note at the end.
A HEPI press release that discusses the findings of their recent survey on students’ attitudes towards drugs makes for interesting reading. There are many useful insights that readers can glean from the data. Most of the analysis clearly reflects students’ responses and I would urge anyone with an interest in this area to read the piece and share the findings, which provide a welcome contribution to the evidence-base.
That being said, I have serious concerns about HEPI’s interpretation of some of their survey responses. As with all survey data, it is worth being cautious when the analysis begins to depart from the precise wording of survey questions – there’s usually a reason for the departure.
This is particularly the case if the analysis in question provides the basis for a headline. For this particular piece, HEPI opted for: ‘Most students think taking illegal drugs causes problems for users as well as society and want their universities to take a tougher stance’. This is a bold and powerful headline and, given the history of the ‘tough on drugs’ rhetoric in popular debate, it is likely to be used by those who wish to pursue more punitive policies towards drugs in and around university campuses. It is crucially important, therefore, that it is supported by the data. My assertion is that it is not.
HEPI’s analysis, on which the headline is based, relies on the following question when raising the issue of a ‘tougher stance’: ‘Do you believe your university should take a stronger line on (i.e. take more seriously)…
- Students who repeatedly use drugs; and
- Drug dealers’
The response options were:
- ‘Yes, they should take it more seriously’
- ‘No, they are already taking it seriously enough’
- ‘No, it is not their role to take it seriously.’
Responding to this survey item, 62% of students opted for: ‘Yes, they should take it more seriously’, in relation to parts 1) and 2). How should these responses be interpreted, and more specifically, is it reasonable to interpret them as supporting HEPI’s headline statement?
Let’s be very clear – there is a reason why the ‘i.e.’ has been inserted into this particular survey question. The term ‘stronger line’ is, of course, ambiguous – an issue that was rightly picked up by those designing the survey. The ‘i.e. …’ was inserted to perform the function of acknowledging and clarifying the ambiguity. The survey item is unequivocally directing respondents to interpret the phrase ‘stronger line’ in the manner explicitly outlined: ‘take more seriously’.
Furthermore, to put the issue beyond doubt, the response options avoid the term ‘stronger line’, opting instead to reiterate the phrase, ‘take it more seriously’. Clearly, therefore, the question is asking respondents if they think that universities ought to ‘take more seriously’ repeated drug users and drug dealers, and 62% think they should. The problem is that HEPI leaps from this particular finding – that 62% of students think universities ought to ‘take more seriously…’ – to the (mis)leading headline: ‘Most students…want their universities to take a tougher stance’.
One of the reasons this is misleading is that ‘take more seriously’ can mean more than one thing. It could mean, for example, a more serious health response (eg additional / better targeted resources in the areas of advice and support) or a more punitive enforcement response (eg a ‘tougher stance’ towards drug users / dealers).
Consider the hypothetical student who believes there should be more advice on the potential risks of taking certain types of drug or another student who believes universities should be providing more support to students who may be experiencing problems associated with drug use. It is entirely reasonable that such students may have ticked the box, ‘Yes, they should take it more seriously’ without those students in any way endorsing a ‘tougher stance’.
Evidence that HEPI’s interpretation is erroneous and misleading is provided by students’ responses to another survey item. Take the following, for example: ‘From the following options, which do you think would be more effective on deterring students from using drugs?’ To this question, just 33% of students selected ‘tougher punishments’, with 41% selecting ‘better education and information…’ (the remaining 26% opted for ‘neither’). Considered in this context, it seems beyond reasonable doubt that a sizeable proportion of the 62% of students responding that universities should take drugs ‘more seriously’ will not have had a ‘tougher stance’ in mind.
Of course, the percentage responses to this latter question (33% and 41% respectively) come as no surprise to someone who can see that HEPI’s interpretation of the responses to the initial question is flawed and misleading. HEPI seem to be clinging to the defence that ‘tougher stance’ is not radically different to ‘stronger line’. Fine. In my opinion, these are substantively different, but this is less crucial than the glaring, primary problem that the question clearly directed students to respond to the phrase ‘take more seriously’.
It is this phrase, ‘take more seriously’, that is not synonymous with ‘take a tougher stance’. If HEPI were originally oblivious to this error of interpretation – thinking that their only mistake was switching from ‘stronger line’ to ‘tougher stance’ – they are not oblivious to it now. To be very clear, the error is not primarily in the framing of the question, although this is far from optimal – it is in HEPI’s own analysis and misleading headline, which distorts the narrative around the survey responses.
It cannot pass without notice that this research was conducted alongside the University of Buckingham, who have recently ‘vowed to become the first [university] in the country to bring in a “drug-free” policy which would force students to sign a contract not to take drugs on the grounds of the institution’. At the very least, it seems plausible that HEPI’s analysis and headline have been influenced – however unconsciously, perhaps – by their partnership with an institution intent on taking a tougher stance on drugs.
This is not a trivial or harmless word game. Unwittingly (or not), HEPI are loading the ammunition of those who will seek to use this research to pursue more punitive drug policies. Indeed, based on HEPI’s press release, media outlets such as The Times ran with the headline: ‘Students call for crackdown on drug use’. This statement was followed up with: ‘Universities have been urged to crack down on drug use on campus by their own students, according to research’. So we have moved from 62% of students responding to the phrase ‘take more seriously’, to ‘students call[ing] for a crackdown on drug use’. If HEPI truly wish to be considered independent and non-partisan, one would hope this would encourage them to be more careful with the language they use when crafting their headlines in future.
Of course, people are free to argue in favour of whatever policies they like. My point is that whatever position is taken, arguments should not be based on poorly constructed, ambiguous survey questions, and, more seriously, inaccurate or misleading interpretations of the evidence that these questions generate.
HEPI have a choice: either they can stand by their principle of being an independent and non-partisan think tank, own their mistake, and correct it – or, they can cling obstinately to an attention-grabbing, misleading headline that does an injustice to those who in good-faith agreed to take part in their survey.
- There are others parts of the survey not mentioned here that need to be considered alongside those questions that are discussed here: for example, in another question over half of the respondents chose ‘No’ when asked ‘Do you think your university does enough to discourage the use of illegal drugs?’ and only 19% said ‘Yes’.
- HEPI has not itself argued for any specific policy responses on the back of the survey.
- For anyone who would like to explore these issues further, we have made the full dataset available to all (as with all our surveys).
I would like to give credit and my respect to HEPI for agreeing to host this blog, despite its critical nature. There aren’t many organisations would be prepared to do this, so for that I am grateful.
As is clear from the blog, I do not think the evidence from HEPI’s survey supports the headline stating that the majority of students want their universities to take a ‘tougher stance’ on student drug-use – that’s why I have challenged their analysis and written this response piece.
In a further (and final) response to the Editor’s Notes that have been added to this blog, here is an additional deconstruction of the relevant survey questions (including Q.7. cited in the Editor’s Notes) demonstrating why I think the headline is built on sand:
Q.10.2 “Do you believe your university should take a stronger line on (i.e. take more seriously): Students who repeatedly use drugs.”
Given HEPI’s article and press release, this is clearly the question on which they primarily rely to support their ‘tougher stance’ comment in the main text and headline.
That is why I focused on this question in the blog. I cannot put it more concisely than this:
When you add an ‘i.e.’ into a survey item, this takes primacy over the meaning of the original term (in this case ‘stronger line’), because you have deliberately and unequivocally directed respondents how they ought to interpret the latter. This is basic linguistics.
As per the blog, ‘take more seriously’ does not equate to ‘tougher stance’.
Q.9 “From the following options, which do you think would be more effective on deterring students from using drugs?”
Of all of the survey questions in this study, this is one of the least poorly worded – and the results are revealing:
A full 67% of students do not think tougher punishments are the most effective way of deterring students.
With this in mind, I would have expected HEPI to hesitate before publishing a headline claiming that a majority of students want a tougher stance, when only 33% selected tougher punishments (and a higher proportion selected ‘better education and information on the dangers of drugs’).
The Editor’s Notes now cite this question as one of their strongest defences:
Q7. Do you think your university does enough to discourage the use of illegal drugs?
The flaw in relying on this question, however, is blatant. Whatever percentage of ‘yes’/‘no’ responses are received, this survey item cannot provide support for HEPI’s headline.
If a student believed that universities ought to do more to discourage drugs by offering more advice on the potential risks associated with certain drugs (see answers to Q9 for a clue on this one), or dedicate additional resources to health professionals on campus to support students to move away from drug use, how do you think that student might respond to this question?
They may well answer ‘No’, but this would not mean they were endorsing a ‘tougher stance’. Therefore, the finding that 52% of students responded with ‘No’ does not provide anything close to a sound basis for the conjured headline and narrative.
A final note to HEPI:
It seems likely to me that this entire piece of research has suffered from confirmation bias from the outset. Conducting it in partnership with the University of Buckingham, a university that earlier this year invited police sniffer dogs onto its campus in a ‘zero-tolerance’ approach to drugs, must surely raise alarm bells around any claim to independence and impartiality (regarding this particular project).
I have read the full list of survey responses and they do not support your conclusions – there’s a very good reason for that: whether you like it or not, (and you clearly don’t), the majority of students do not want their universities to take a ‘tougher stance’ on student drug-use.
They may well want their university to take drugs ‘more seriously’ – whatever that means (I’m afraid we can’t tell because the survey was badly designed) – but there is no evidence that most are calling for a ‘tougher stance’.
To reiterate, unwittingly or not, I strongly believe you are doing an injustice to the trust and good faith of those who agreed to take part in your survey by misrepresenting their responses.
Drug policy is a serious issue – I care passionately that it is based on the best possible data and on a sound evaluation of the evidence.
If you would like to conduct further studies in this area, I would be happy to support you to design a piece of research that is capable of informing (rather than distorting) the debate around drug policy.
I do not expect that offer will be taken up, but in any case, I would like to thank you again for engaging openly with me following the publication of your report and press release.
Lecturer in Criminology
The Open University