Placebo effect in RCTs: A ‘theory of active research participation’
The Theory of Active Trial Participation, Scott et al. (forthcoming)
A forthcoming paper in Social Science & Medicine studies influences of participation in and outcomes of a randomized controlled trial. From the abstract:
The results indicate that trial recruitment and retention depend on a set of convictions forged largely as a result of contextual factors peripheral to the intervention, including the friendliness and helpfulness of research centre staff and status of the administering practitioner. These convictions also influence the reporting of the study outcomes, particularly if participants experience uncertainties when choosing an appropriate response. The findings suggest that participants in clinical trials are actively involved in shaping the research process, rather than passive recipients of treatment. Thus the outcomes of trials, notably those involving contact interventions, should be regarded not as matters of fact, but as products of complex environmental, social, interpretive and biological processes.
Though the qualitative study was embedded in a clinical trial on acupuncture, I couldn’t help but consider the same problem as posed against randomized controlled trials (RCTs) used to test the effects of policy interventions. There is a particular characteristic of the acupuncture study that is similar to policy intervention RCTs: the fact that they’re not double-blind. Policy RCTs cannot control for placebo effects. The forthcoming article is a great introduction to RCTs, and discusses double-blinding:
Provided the test and control interventions are comparable and attrition distributed evenly across groups, randomisation eliminates selection bias and controls for placebo effects. ‘Double-blind’ means that neither the patient, nor the attending practitioner, nor the researchers who analyse the data know which intervention the patient has received (Miller and Stewart, 2010). Blinding of the practitioner is only possible if the test and control treatments are indistinguishable at the point of administration, as in most drug trials.
Because I have yet to come across a policy intervention RCT that is double-blind, I suspect such interventions could be plagued with similar influences that cannot be attributed to the treatment, but to the study itself.
I’ve been trying to think about how we might test for this… but find it kind of silly to design an RCT to test the effects of an RCT, especially when we should be shorting RCTs altogether.
haba na haba 
Don’t worry, the economists are on it – here’s a paper published in February which finds and measures a “placebo effect” of just being surveyed on later behavior. http://www.pnas.org/content/108/5/1821.full.pdf+html
But in addition to the Hawthorne Effect, I wonder what happens when researchers have lotteries, perhaps suggesting there is something to be won (e.g. Blattman’s Randomization in the Tropics). We could expect that lottery “winners” of the treatment arm might be more convicted to have better outcomes and lottery “losers” (in the control arm) might feel even more disappointed about not receiving the treatment, which could give them even stronger survey responses in the future about the lack of progress in the policy area which the randomized intervention was meant to impact.
“Because I have yet to come across a policy intervention RCT that is double-blind, I suspect such interventions could be plagued with similar influences that cannot be attributed to the treatment, but to the study itself.”
Except placebo effects are generally the result of the subject not being certain whether or not he/she is receiving a treatment. Most policy RCTs are transparent in this manner – you know if your kid got dewormed, you know if you received some money. Double-blind testing *drives* placebo effects, not solves them.
You can, as Lee says, get surveyed effects (although I’d argue these are pretty hard to convincingly detect), but we’d have no reason to suspect these would be different, in level, between treatment and control groups.
The remaining problem is the Hawthorn effect, where the treatment group responds in a more positive manner because they know they are the target of further research, but this is something distinct from placebo effects.
Yes, Matt — the original post as I wrote it was titled “Hawthorne Effect”, which is indeed different from placebo effects (but read a bit “inside baseball” to me, so I hastily changed it).
Certainly, people are more likely to know if they are in the treatment arm for individual-level policy interventions (i.e. deworming, cash transfers), but my suspicion is ordinary people are less likely to know if they are in the treatment arm for higher-level policy interventions (i.e., water source improvements randomized at the village or cluster level, curriculum changes randomized at the district level).