Ecstasy for PTSD: Not Ready for Prime Time
sciencebasedmedicine.org
sciencebasedmedicine.org
"The study is characterized as blinded; but the blinding didn’t work, since 95% of subjects were able to tell which group they were in."
As the researchers themselves say, the purpose of the study was to test their methodology before doing larger trials, including whether or not their blinding protocols were effective. They are going to be using active placebos for phase III, which are much more effective for blinding. In fact many of the other studies they've done already use active placebos, either a lower dose of MDMA or else something like methylphenidate. The authors address this in the actual study and explain that they now have FDA approval to use a three-arm protocol in the future, but SBM is pretending that this a methodological problem they found themselves.
"There is only a pilot study with scanty, preliminary, unconfirmed evidence."
There have been at least a dozen studies looking at MDMA as a treatment for PTSD, but again the author is pretending that this is the only one.
"Experiments using illegal drugs are always problematic, because researchers tend to be advocates for legalization and may be biased by their emotional investment."
Most of the MDMA studies use a triple blind methodology which is far more rigorous than the methodology that big pharma uses. What this means is that not only are the patients and the people administering the drug blinded (or will be going into phase III trials), but also the people analyzing the data were not the ones administering the drugs so they have no idea who was in each group.
Also, these are multi-site trials going on in various countries around the world, including ones where the legal status and cultural baggage surrounding MDMA is wildly different in both researchers and the general population.
"The patients in the placebo group improved, too; so a large part of the benefit could be attributed to the psychotherapy."
As one would expect. However, they still had PTSD whereas the people who got the MDMA didn't. Additionally, studies have shown that the MDMA group continues to improve in the months and years after therapy much more than the control group.
"The addition of MDMA to psychotherapy might prove helpful in refractory cases of PTSD, but the preliminary results of this one small pilot study will need to be confirmed by larger studies in combat veterans before the treatment can recommended to patients."
Agreed. Except for that again there have been many studies of MDMA as a treatment for PTSD, not just this one.
"Unfortunately, the history of medicine is full of equally promising treatments that didn’t pan out. Probably the most typical course is this: a strongly positive pilot study is followed by larger studies with weaker positive results, then by studies with equivocal or negative results, then reports of serious side effects or other problems, and eventually by rejection of the treatment. I sincerely hope this one will prove an exception to that course, for the sake of suffering veterans with few remaining options. But I am not optimistic."
The reason why this has happened consistently in medicine is because the studies are being funded by for profit companies and they are using extremely biased methodology to make their drugs look much better than they actually are. Whereas MAPS is a non-profit, and their methodology is designed to be as neutral as possible. The research may well still not pan out as trial sizes increase and patients are more effectively double blinded, but there is definitely reason for optimism based on the results so far.
For what it's worth, there are numerous videos online where you can watch the researchers (and MAPS) talking about these studies:
http://www.maps.org/videos/source/video3.html (This video is with Michael Mithoefer, one of the researchers behind this study.)
http://vimeo.com/32062578 (Talk given by MAPS about the roadmap for getting MDMA approved.)
There are several more videos online here about both the mechanisms of action, the pharmacology, and the results of other trials:
Cheers.
Personally though I think watching the lectures by the researchers (and the MAPS staff) is a better way to learn about this stuff than reading the actual research. Normally I prefer just reading the primary source, but when you have the actual researcher explaining each graph in their powerpoint then it's hard to beat that. (Albeit it is somewhat scattershot compared to reading the entire literature.)
If a for-profit company pushes a study through a first stage trial and then loses out at a later stage they lose much more money than if they had given up quickly. What's really necessary for most promising early stage treatments to not pan out is for genuinely good treatments to be rare, plus people being more likely to report impressive results than failures. Both of which I hope you'll admit are just as much a problem for not-for-profit researchers as for-profit ones.
I agree that genuinely good treatments are rare. However, the file drawer problem is definitely much worse for industry-sponsored studies than for, say, non-profit- or NIH-sponsored ones. MAPS has a tiny budget of less than a million dollars a year, and if they were to do a study and not publish the results they would get completely discredited. And for an NIH researcher, if they didn't publish their results then it's unlikely they'd ever get more grants in the future. I don't know of any specific stats about how the file drawer effect for NIH-sponsored studies compares to industry sponsored ones, as that would be the closest comparison. But I do have some quotes that address the issue in general, and explain why the problem is much worse with for-profit companies:
"The failure to publish unsuccessful trials presents a problem in many research areas. When a study has produced non-significant results, it is less likely to be submitted for publication; and, if it is submitted, it is less likely to be favourably reviewed or accepted for publication. But although publication bias affects all areas of research to some extent, it is particularly acute when it comes to drug trials. This is because most of the clinical trials evaluating medications are financially sponsored by the companies that produce and stand to profit from them. The companies own the data that come out of trials they sponsor, and they can choose how to present them to the public -- or to withhold them and not present them to the public at all. With widely prescribed medications, billions of dollars are at stake. In this case, it is not reviewers or journal editors who are impeding publication of negative findings. Rather it is the companies themselves that decide to withold negative data from publication. [...]
Most studies showing negative results remain unpublished, and short of making official enquiries to government agencies, their data are unavailable to researchers, doctors and the public at large." Source: The Emperor's New Drugs, p. 39
"The fact that what gets published are the trials with positive results was most convincingly shown by a group of researchers at the Oregon Health and Science University, who followed up on our initial analysis of the FDA data by comparing the conclusions reached by the FDA with those reported by the drug companies in journal articles. Of 38 drug-company clinical trials that the FDA viewed as having positive results, all but one was published. In the same documents, the FDA described 36 trials as having negative or questionable results. Most of these negative trials were not published at all, and of the few that were published, most were described in the journal articles as showing positive results -- despite the fact that the FDA had concluded that they had not." Source: The Emperor's New Drugs, p. 67, describing the journal article<a href="http://www.nejm.org/doi/full/10.1056/NEJMsa065779>Select... Publication of Antidepressant Trials and Its Influence on Apparent Efficacy</a>
"When application was made to the Swedish Drug Authority for approval of five new antidepressant drugs, 28 separate clinical trials evaluating the drugs' effectiveness had been published in medical journals. The results were overwhelmingly positive: Twenty-two studies showed that the new drugs were significantly more effective than a placebo, and only 6 showed no difference. In Sweden, drug applications must including <i>all</i> known studies -- published or not -- relevant to the new drug. When researchers from the Swedish Drug Authority went through the new drug applications for the five new antidepressants, they found that a total of 42 studies had been completed. It turned out that exactly half of these showed that the new antidepressants are more effective than placebos and half found that they are not. The 22 postive articles that had been published represented 19 of the positive studies (three were published twice). In contrast, only six of the 21 studies with negative or inconclusive findings had been published. Even the most conscientious doctor could know only the results of the studies that had been published and would reasonably conclude that the weight of the evidence about the new antidepressants was overwhelmingly positive.
The Swedish researchers commented that their finding that 40 percent of the studies that had been completed on these drugs remained unpublished (as independent studies, not pooled with others) was consistent with the findings of other such reviews. In their conculsion, they warned that 'for anyone who relies on published data alone to choose a specific drug, our results should be a cause for concern.... Any attempt to recommend a specific drug is likely to be based on biased evidence.' What else can a practicing physician rely on but the published data? In an understated way, these researchers were telling doctors that they could not trust the published scientific evidence about antidepressants to be complete and unbiased." Source: Overdosed America, p. 115
"Whereas many of the negative trials were not published at all, some of the positive trias were published many times, a practice known as 'salami slicing', and this was often done in ways that would make it difficult for reviewers to know that the studies were based on the same data. In some cases, the authors were different, and references to previous publication of the data were often missing. Sometimes there were minor differences in the data between one publication and another, as well as between the data as presented to regulatory agencies and the data as published. So a reviewer trying to summarize the data would be likely to count the positive data more than once.
Another trick was to publish only some of the data from a clinical trial, a manoeuvre that researchers call cherry-picking the data. Some clinical trials are conducted in more than one location. These are called multi-centre studies. Multi-centre studies make it easier to find sufficient patients to conduct the trial. They also make it easier to cherry-pick the data. For example, one multi-centre study of Prozac was presented to the FDA as showing a drug-placebo difference of three points on the Hamilton scale. When the data from this clinical trial was published, the difference was reported as 15 points -- a five-times increase in effectiveness. How was this magical augmentation of the benefits of Prozac accomplished? The full study was conducted on 245 patients. The published paper reported data from only 27 of these patients. In the published version, the data from the bulk of the patients were left out, making the drug seem much more effective than it really was." Source: The Emperor's New Drugs, p. 41
"Drug companies also publish 'pooled analyses' of the trials they have conducted. That is, they bundle together the results of different trials and analyse the drug-placebo difference across them. This is similar to the meta-analyses my colleagues and I have conducted, but with one important difference. Our meta-analyses, in common with most others reported in the scientific literature, are based on all the studies that we were able to find. In contrast, the drug companies pick and choose which studies they wish to include in their pooled analyses. For example, GlaxoSmithKline submitted 15 clinical trials of Seroxat to Swedish regulators. In addition to being published individualy -- sometimes more than once -- studies with positive results were also included in six different pooled analyses. Most of the studies with negative results were, of course, not included in the pooled analyses.
There is yet another way in which pooled analyses can hide negative data. Rather than not publishing the negative data at all, the companies can bundle them together with data from positive trials, so that the overall result is positive. By so doing, they can truthfully claim that they have published the data from a negative trial, while hiding the fact that those data showed no difference between a drug and placebo. The article by the Swedish regulators showed that the data from about 20 per cent of trials were not published at all. The data from another 20 per cent of trials were bundled together with data from more successful trials, so that their negative results were hidden from view. Taken together, approximately 40 per cent of the data are kept out of sight. Practices like this [...] make it exceptionally difficult for reiewers to establish how effective the drugs really are." Source: The Emperor's New Drugs, p. 42
"Studies repeatedly document bias in commercially sponsored research, but the medical journals seem powerless to control the scientific integrity of their own pages. In 2003, separate studies were published in JAMA and the <i>British Medical Journal</i> showing that the odds are 3.6 to 4 times greater that commercially sponsored studies will favor the sponsor's products than studies without commercial funding. And in August of 2003 a study published in JAMA found that among the highest-quality clinical trials, the odds that those with commercial sponsorship will recommend the new drug are 5.3 times greater than for studies funded by non-profit organizations. The authors noted that the lopsided results of commercially sponsored research may be 'due to biased interpretation of trial results.' They cautioned that readers should 'carefully evaluate whether conclusions in randomized trials are supported by data.' In other words, doctors are warned that even the best research published in the best journals <i>cannot</i> be taken at face value." Source: Overdose America, p. 97. Citing <a href="http://www.ncbi.nlm.nih.gov/pubmed/12533125>Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A systematic Review</a>, <a href="http://www.bmj.com/content/326/7400/1167.full>Pharmaceut... Industry Sponsorship and Research Outcome and Quality: A Systematic Review</a>, <a href="http://jama.jamanetwork.com/article.aspx?articleid=197132... of Funding and Conclusions in Randomized Drug Trials</a>
"In the case of calcium channel blockers like Norvasc, for instance, one survey of seventy articles about their safety found that 96 percent of authors who were supportive of the drugs had financial ties to the companies that made them, whereas only 37 percent of authors who were critical had such ties." Source: The Truth About Drug Companies, p. 107, citing <a href="http://www.nejm.org/doi/full/10.1056/NEJM199801083380206>... of Interest in the Debate over Calcium Channel Antagonists</a>
"A team of reserachers at the Beth Israel Medical Center in New York have examined the outcome of clinical trials as a function of who had sponsored them. They found that approximately 75 per cent of drug-company studies showed favourable results for their own drugs, but only 25 per cent of them showed favourable results for the product of a competing company. In studies that are not sponsored by a drug company, the success rate is approximately 50 per cent." Source: The Emperor's New Drugs, p. 62, describing <a href="http://journals.cambridge.org/action/displayAbstract;jsessio... between drug company funding and outcomes of clinical psychiatric research</a>
See also:
And weed cures all forms of cancer -- that's why it's illegal.
If Big Pharma wanted to push dangerous or ineffective drugs on the public, there are far easier and cheaper ways to go about it than coming up with elaborate double-blind, randomized, placebo-controlled testing protocols. See: Vioxx.
Experiments using illegal drugs are always problematic, because researchers tend to be advocates for legalization and may be biased by their emotional investment.
But MAPS, a pro-psychedelics organization, is neutral. Shyeah -- about as neutral as High Times.
This person is expressing sincere doubt -- probably because she's seen enough medical research to know that drugs that seem like miracle cures in pilot studies often pan out to be not so miraculous when tested in broader studies with larger cohorts -- and that's true whether the drug in question is politically controversial (like MDMA, LSD, or marijuana) or not.
I say this as one who supports drug legalization and wants to see further research conducted with MDMA. Until the results of larger studies come in, there's still room for doubt here.
That's demonstrably false. There are dozens of books on the topic, many of which I link to here:
http://news.ycombinator.com/item?id=4807436
"Until the results of larger studies come in, there's still room for doubt here."
No question there's room for doubt. My issue isn't with the fact that the author doubts that MDMA will be effective, which it may well not be, but rather that she is being intellectually dishonest in her arguments.
Fortunately, the economic disincentive is getting larger each year:
http://en.wikipedia.org/wiki/List_of_largest_pharmaceutical_...
You haven't demonstrated it false; I can demonstrate it true.
she is being intellectually dishonest in her arguments.
Or she is just unaware of the scope of MDMA reesearch conducted thus far. I doubt this is her specialty.
??? Vioxx was not at all cheap or easy to bring to market, nor was it ineffective. Rather, it was a case where a drug was tremendously beneficial to most users but deadly for a small subset. Even after its withdrawal from the market, US and Canadian government panels have voted in favor of bringing it back based on cost-benefit analysis!
http://en.wikipedia.org/wiki/Vioxx#Withdrawal
I don't think it's possible to have a worse example of the drug company behavior you mentioned. Why not point to the days of phony, deadly cures being pushed on a naive population? Vioxx had actual research and efficacy behind it, and may actually merit coming back on the market.
Why not point to the days of phony, deadly cures being pushed on a naive population?
That's still going on, except these days the pushers of phony, deadly cures sound a lot like Alex3917. Hint: look for accusations of medical researchers being "shills of big pharma".
Vioxx gives us a handy, recent example of big pharma actually behaving badly,and they had to undermine science to do it. Just like the peddlers of unproven "alternative cures" -- and quite unlike, and to the dismay of, the medical research community.
http://en.wikipedia.org/wiki/Serotonin_syndrome
It can take weeks to a month of widthdrawl from SSRI to prevent interactions.
Doctors will try and fix the depression, but then something else will "arrive" and wreck havoc. Whether that's anxiety, flashbacks or something else that's not "always there" when you're simply "just" depressed.
It's nearly impossible to treat PTSD as a whole with conventional medicine.
> The most commonly prescribed class of medications for PTSD (and the one approved by the U.S. Food and Drug Administration) are the selective serotonin reuptake inhibitor (SSRI) antidepressants.
http://psychcentral.com/lib/2006/an-overview-of-treatment-of...
Contrary to several comments posted while I was busy, the edited group blog Science-Based Medicine is about applying the methods and settled factual conclusions of science to the existing practice of "evidence-based medicine." As the group of authors put it, "Science-Based Medicine is dedicated to evaluating medical treatments and products of interest to the public in a scientific light, and promoting the highest standards and traditions of science in health care."
http://www.sciencebasedmedicine.org/index.php/about-science-...
There have been a variety of posts on the site over the years critical of large pharmaceutical companies, and there have been a variety of posts critical of various aspects of current medical practice. The tenor of the site is that everyone involved in treating human patients and guarding their health needs to be truth-seeking, considering prior probabilities and scientific fact when evaluating new treatments. The authors are not a propaganda outlet for any industry, and they have no financial conflicts of interest on the issues they write about.
Dr. Hall correctly points out that "Prolonged exposure therapy is a widely recognized therapy for PTSD. The rationale for using MDMA is to suppress anxiety and allow patients to talk about their traumatic experiences without feeling excessive fear and hyperarousal." Any effort to find a new treatment for PTSD, whether a prescribed drug or a new form of talk therapy, will have to be evaluated against what is already available to treat that tough disorder.
Any reader of Hacker News who hasn't seen the link below already should definitely check out LISP hacker (and Google director of research) Peter Norvig's article "Warning Signs in Experimental Design and Interpretation,"
http://norvig.com/experiment-design.html
which is a great checklist of what to look for in a report of a new research finding. Dr. Hall knows how to evaluate research reports in this framework, and she urges that people (like her) who hope that more effective treatments for PTSD are developed proceed carefully so that the new treatments are truly safe and effective.
This pins the WTF meter. 1 out of the 20 could not figure it out?
In related info: The controlled studies for viagra were a bit comical.
Suggestion is a powerful thing to some folks.
This is discussed in more detail in Irving Kirsch's The Emperor's New Drugs.
Antidepressants tend to be long lasting and slow acting (AFAICT) which MDMA is not.
Regarding the issue of whether or not it is poor blinding, I guess it is a bit like the tree falling in the forest but nobody is around to hear it: if you conduct a placebo-controlled study, but all the participants can correctly guess which group they're in, is it really a placebo-controlled study?
I guess (as may have been suggested elsewhere) you could use a non-seratonergic stimulant of some sort to give people the feeling that they are on something, and rely on them not being able to tell what it is. You'd need to screen for anyone that had any recreational experience though.
Speed would probably work. One thing that interests me though is whether or not the patients were treated seperately. Having been surrounded by people on MDMA while not partaking myself, at least in my anecdotal experience it can affect your state of mind, which could be one reason for the rather large placebo effects (given that the treatment is so obvious, one would expect placebo effects to be artificially lowered.
No, it's not, and the question of how many allegedly double-blind studies are compromised by this leakage deserves a lot more attention.
The page also mentions that the reputation of MDMA preceded it and may have had an impact on the study. Using a less well known substance may have tackled this.
Not to mention if you can ensure you're getting the right thing and not some dodgy multiply-substituted cathinone or piperazine!
There is very little evidence that MDMA at therapeutic doses causes any more damage than all of the other legal prescription drugs like prozac.
No kidding