- Jul 2018
-
europepmc.org europepmc.org
-
On 2015 May 17, Tim Smits commented:
Below is a Letter submitted to the Lancet concerning this publication. Lancet did not accept this Letter, though it addresses a critical aspect of the preregistration process (well, actually it was postregistration). For a visual timeline of the errors in the preregistration process, I refer to a blogpost http://persuasivemark.blogspot.be/2014/03/the-pitfalls-of-pre-registration.html. Such major errors in the preregistration do make the benefits of such quality control measures obsolete and could provide a cover-up for researcher degrees of freedom.
In their article on non‐medicated cognitive therapy for schizophrenia, Morrison and colleagues[1] report on significant reductions in psychiatric symptoms. These interpretations are rash, due to two severe limitations. The protocol for the study is difficult to find and reconstruct. The lead author registered[2] the trial on October 21st 2010, about eight months after data collection started, without making reference to planned analyses. An article with a more extensive protocol was published[3] in 2013. It was first submitted in October 2012, thus compromising the 9 to 18‐month study duration ending in February 2013. That article did, however, include more detailed analysis plans: “…analysis of repeated measures using a mixed‐effects model…”. Although the proposed testing of treatment effects at the individual level seems the best approach, the final Lancet article only reports effects between treatment and control groups at different time points. Clearly, this is an inferior analysis of a longitudinal repeated‐measures design, and it violates the aforementioned protocol[3] . The published analyses fail to take into account interpersonal variability at the onset of data collection. They also fail to capitalize on the design’s capability to identify individual treatment effects.<br> Ironically, Morrison and colleagues claim to report on all outcomes specified in their protocol, but they fail to report on their planned analyses. Such conduct increases researcher degrees of freedom, while simultaneously obscuring the use of this freedom with the protocol registration allegedly backing up the reported research practices. As others have claimed previously[4,5], the implications of such practices are potentially serious.
References
1 Morrison AP, Turkington D, Pyle M, Spencer H, Brabban A, Dunn G, Christodoulides T, Dudley R, Chapman N, Callcott P, Grace T, Lumley V, Drage L, Tully S, Irving K, Cummings A, Byrne R, Davies LM, Hutton P. Cognitive therapy for people with schizofrenia spectrum disorders not taking antipsychotic drugs: a single‐blind randomised controlled trial. Lancet 2014 ahead of print
2 http://www.controlled‐trials.com/ISRCTN29607432/morrison
3 Morrison AP, Wardle M, Hutton P, Davies L, Dunn G, Brabban A, Byrne R, Drage L, Spencer H, Turkington D. Assessing Cognitive Therapy Instead Of Neuroleptics: Rationale, study design and sample characteristics of the ACTION trial. Psychosis 2013; 5(1): 82‐92.
4 Schulz KF, Altman DG, Moher D. Protocols, probity, and publication. Lancet 2009; 373: 1524.
5 Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014; 383: 267‐76.
This comment, imported by Hypothesis from PubMed Commons, is licensed under CC BY. -
On 2014 Apr 17, James C Coyne commented:
This abstract is exceedingly misleading for a study that was registered after enrollment started and without adequate designation of which time point which outcome would be assessed as primary. At the end of the intervention, there were no significant differences between CBTp and treatment as usual.
In 2 blog posts at PLOS Mind the Brain, I discuss the serious problems with this study:
The trial really did not producing usable data concerning the efficacy of cognitive behavior therapy for patients with unmedicated schizophrenia because of
An unusually mixed group of patients participating in the study. An inappropriately constructed control group that does not represent conditions in routine care nor as a composite allow meaningful comparisons with the active intervention, CBTp. Substantial loss to follow-up from an already small exploratory study. A decision of the investigator team to abort long-term follow-up but proceed with data analysis as if this decision had not been made. A substantial number of patients in both the intervention and control group receiving antipsychotic medication, including those in the control group who showed the greatest improvement.I could go on but best to see these critisms and others elaborated with others at my blog posts.
This comment, imported by Hypothesis from PubMed Commons, is licensed under CC BY.
-
- Feb 2018
-
europepmc.org europepmc.org
-
On 2014 Apr 17, James C Coyne commented:
This abstract is exceedingly misleading for a study that was registered after enrollment started and without adequate designation of which time point which outcome would be assessed as primary. At the end of the intervention, there were no significant differences between CBTp and treatment as usual.
In 2 blog posts at PLOS Mind the Brain, I discuss the serious problems with this study:
The trial really did not producing usable data concerning the efficacy of cognitive behavior therapy for patients with unmedicated schizophrenia because of
An unusually mixed group of patients participating in the study. An inappropriately constructed control group that does not represent conditions in routine care nor as a composite allow meaningful comparisons with the active intervention, CBTp. Substantial loss to follow-up from an already small exploratory study. A decision of the investigator team to abort long-term follow-up but proceed with data analysis as if this decision had not been made. A substantial number of patients in both the intervention and control group receiving antipsychotic medication, including those in the control group who showed the greatest improvement.I could go on but best to see these critisms and others elaborated with others at my blog posts.
This comment, imported by Hypothesis from PubMed Commons, is licensed under CC BY. -
On 2015 May 17, Tim Smits commented:
Below is a Letter submitted to the Lancet concerning this publication. Lancet did not accept this Letter, though it addresses a critical aspect of the preregistration process (well, actually it was postregistration). For a visual timeline of the errors in the preregistration process, I refer to a blogpost http://persuasivemark.blogspot.be/2014/03/the-pitfalls-of-pre-registration.html. Such major errors in the preregistration do make the benefits of such quality control measures obsolete and could provide a cover-up for researcher degrees of freedom.
In their article on non‐medicated cognitive therapy for schizophrenia, Morrison and colleagues[1] report on significant reductions in psychiatric symptoms. These interpretations are rash, due to two severe limitations. The protocol for the study is difficult to find and reconstruct. The lead author registered[2] the trial on October 21st 2010, about eight months after data collection started, without making reference to planned analyses. An article with a more extensive protocol was published[3] in 2013. It was first submitted in October 2012, thus compromising the 9 to 18‐month study duration ending in February 2013. That article did, however, include more detailed analysis plans: “…analysis of repeated measures using a mixed‐effects model…”. Although the proposed testing of treatment effects at the individual level seems the best approach, the final Lancet article only reports effects between treatment and control groups at different time points. Clearly, this is an inferior analysis of a longitudinal repeated‐measures design, and it violates the aforementioned protocol[3] . The published analyses fail to take into account interpersonal variability at the onset of data collection. They also fail to capitalize on the design’s capability to identify individual treatment effects.<br> Ironically, Morrison and colleagues claim to report on all outcomes specified in their protocol, but they fail to report on their planned analyses. Such conduct increases researcher degrees of freedom, while simultaneously obscuring the use of this freedom with the protocol registration allegedly backing up the reported research practices. As others have claimed previously[4,5], the implications of such practices are potentially serious.
References
1 Morrison AP, Turkington D, Pyle M, Spencer H, Brabban A, Dunn G, Christodoulides T, Dudley R, Chapman N, Callcott P, Grace T, Lumley V, Drage L, Tully S, Irving K, Cummings A, Byrne R, Davies LM, Hutton P. Cognitive therapy for people with schizofrenia spectrum disorders not taking antipsychotic drugs: a single‐blind randomised controlled trial. Lancet 2014 ahead of print
2 http://www.controlled‐trials.com/ISRCTN29607432/morrison
3 Morrison AP, Wardle M, Hutton P, Davies L, Dunn G, Brabban A, Byrne R, Drage L, Spencer H, Turkington D. Assessing Cognitive Therapy Instead Of Neuroleptics: Rationale, study design and sample characteristics of the ACTION trial. Psychosis 2013; 5(1): 82‐92.
4 Schulz KF, Altman DG, Moher D. Protocols, probity, and publication. Lancet 2009; 373: 1524.
5 Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014; 383: 267‐76.
This comment, imported by Hypothesis from PubMed Commons, is licensed under CC BY.
-