From the New England Journal of Medicine on randomized clinical trials. You would expect one of the most well-known medical journals in the world to be in favor of clinical trials, but that doesn’t quite seem to be the case. The article is very much a on-the-one-hand on-the-other-hand affair:
By the turn of the 21st century, RCTs had achieved the status of gold standard for therapeutic evidence — but one with well-documented limitations. Physicians continue to pursue alternative methods of knowledge production that are faster or less expensive than RCTs, or that claim to answer questions that RCTs cannot. Yet beyond medicine, RCTs are increasingly emulated, even idealized. . .
Yet despite their limitations, RCTs have revolutionized medical research and improved the quality of health care by clarifying the benefits and drawbacks of countless interventions. Clinical investigators, supported by government funding and empowered by FDA regulations, have used RCTs to advance clinical research theory and practice. Critics have become increasingly adept at ferreting out flaws in RCTs, forcing trialists to be more vigilant in their designs. From a historical perspective, the RCT is not a single or stable technique, but an approach that has evolved as physicians have revised and refined clinical research.
The idea that RCTs would be the only authoritative arbiter to resolve medical disputes has given way to more pragmatic approaches. Experimentalists continue to seek new methods of knowledge production, from meta-analyses to controlled registry studies that can easily include large numbers of diverse patients. Observational methods are seen as complementary to RCTs, and new forms of surveillance can embed RCTs into the structure of data collection within electronic health records. RCTs are now just a part — though perhaps the most critical part — of a broad arsenal of investigative tools used to adjudicate efficacy and regulate the therapeutic marketplace. This status may continue to evolve with the recent turn (back) to personalized or precision medicine. As medicine focuses on the unique pathophysiology and coexisting conditions of individual patients, the applicability of the generalized data produced by RCTs will come under intensified scrutiny.
The whole article is like that – trials have done great things, but there’s just something not right about them, somehow. I can’t quite get a handle on where the authors are coming from, and I don’t seem to be the only reader with that problem. Here’s Vinay Prasad’s take, and he seems to think this is part of a larger movement at the NEJM:
This week the NEJM published, “Assessing the Gold Standard — Lessons from the History of RCTs.” The article rather boringly describes the history of RCTs, and makes some uncontroversial points, but at the same time it systematically denigrates the role of the RCTs and undermines their importance to justify future medical treatments.
Most of the criticisms of RCTs made in the article are completely in error. For this reason, the article is a subtle threat to the pursuit of evidence-based medicine and a threat to better decisions for patients. And, precisely because it is subtle, its danger is all the greater. It is likely to be swallowed, hook line and sinker. Furthermore, the article comes at a time when the fundamental editorial direction of the Journal has been questioned. This article is likely further evidence of the NEJM’s regressive thinking, and is a strategic move by the Journal to undermine evidence.
Why on earth would they want to do that, is the question. The only thing I have to offer, and I really hope that I’m wrong about this, is that some of the authors of this current paper are history-of-science and history-of-medicine types. David Wootton’s The Invention of Science (blogged about here) goes into detail on some of the battles that have gone on in those fields. There have been acrimonious disputes about what science really is, what ways we have of knowing what the facts really are, and what exactly a fact is in general. As you can see, things get pretty philosophical, and they also get pretty political, too, with all sorts of stuff being dragged in from the social science hallways about privileged ways of knowledge, etc.
I really wonder if some of that is behind this latest article, especially when I read in it passages like this one:
Such controversies attracted attention from social scientists and policy scholars. As sociologist Steven Epstein noted, RCTs had become “crucial sites for the negotiation of credibility, risk, and trust.” When they take place in fraught medical, social, and political contexts, RCTs, “rather than settling controversies, may instead reflect and propel them.”46 Historian Harry Marks argued that RCTs must be understood not merely as scientific techniques but also as social events: “even the simplest RCT is the product of a negotiated social order, replete with decisions — some contested, some not — and with unexamined assumptions.”36 Even though RCTs were developed to produce generalizable, universal biomedical knowledge, they have remained deeply entangled in local social conditions, economics, and politics.
That, to me, is the social science worldview in a nutshell – that everything, simply everything, is deeply entangled in social conditions, economics, and politics. Take an NMR spectrum? A political act. Weigh out some copper sulfate? Politics. I sometimes think that that’s my vision of Hell. A more immediate vision of it, for me, can be found in the sorts of abstracts tweeted out by RealPeerReview (a recent example of which made the pages of Science is here). Behind some of this stuff is a worldview that holds that there isn’t such a thing as “knowledge” at all, just power struggles and wishful thinking. It’s hard for me to put into words just how strongly I reject that line of thought, which at one point seemed to be going too far for even some of its practitioners. Inasmuch as I can follow what he’s talking about, Bruno Latour seems to be one of the leading writers in this area who, in more recent years, has been wondering just exactly what he’s done.
I hope that’s not what’s going on here. The NEJM is too valuable as a medical journal to head down that road. For now, have a look at the article and at Prasad’s detailed critique of it. He says the journal is on a mission to “further its agenda to replace better evidence with lesser evidence“, and I hope he’s wrong.