On 2017 May 03, Doug Berger commented:

MISTAKEN TITLE: THE COGNITIVE THERAPY GROUPS WERE NOT BLINDED & HAD NO BLIND-PLACEBO ARM

I thank Dr. Jarrett for replying, however, the issue of the misleading title is not resolved because the cognitive therapy arms are neither single- nor double-blinded, and neither is there a blind-placebo control for the cognitive therapy arms. Thus, it is a violation of clinical trial logic to compare the double-blinded medication arms to the unblind psychotherapy arms in this trial as the data is born from completely different designs. The authors would be well-respected to admit this serious infraction of trial logic and retract the paper (perhaps they were not well aware of this problem 20 years ago).

It is impossible to filter-out the effects of hope and expectation that may bias results in a condition such as depression that has subjective endpoints and large random error. Non-blind outcome research is only excusable in conditions with objectively measurable biological endpoints such as bone fracture rates, myocardial infarction, stroke, tumor size etc.

It is known that 30% of patients receiving drug or psychotherapy placebo can be deemed “responder” in depression. Even a small amount of bias can add up when a study is unblinded as in a psychotherapy so that a large N can make a biased study seem significant. Jarrett has fallen on the slippery slope of opining that blind-raters can make up for some kind of bias in the subject-treater system (unblind treaters also impart hope and expectation to the patient), raters only rate what the subjects’ report.

Antidepressants on the market were required to prove efficacy in double-blind conditions against blind-placebo per FDA regulation-there is no agency that directs the rigor of a psychotherapy outcome trial. The hurdle to do this (a superiority study) is much higher than the non-inferiority results that Jarrett notes as “did not differ”. Comparing blind medication and unblind psychotherapy in the same study severely handicaps the medication arms and thus invalidates clinical trial logic.

If one wishes to look at meta analyses, this one here found only small treatment effects for cognitive behavioural therapy in Major Depression when controlled for blinding: Lynch D, Laws KR, McKenna PJ: Cognitive behavioural therapy for major psychiatric disorder: does it really work? A meta-analytical review of well-controlled trials. Psychol Med. 2010; 40(1): 9–24.

Regardless of quoting meta analyses, there is no way to get around the issue of the inability to blind or have a blind-placebo comparator and thus the inability to filter out hope and expectation in subjects with subjective conditions such as a depressive disorder. The inability to blind psychotherapy trials is part of a larger problem in cognitive behavioural therapy, derivatives such as DBT, as well as other psychotherapy outcome research. Unfortunately both professionals and the lay public alike mistakenly believe that "randomized" and "blind raters" indicates rigorous study for depression (and other psychiatric conditions) as they are not aware of the lack of double-blinding.

The other points in this post are all repeated and referenced in the article noted below: https://www.ncbi.nlm.nih.gov/pubmed/26870318.2

On 2017 May 03, Robin B. Jarrett, PhD commented:

This randomized controlled trial evaluated the extent to which 10-weeks of acute phase phenelzine plus clinical management or cognitive therapy reduced depressive symptoms compared to pill placebo plus clinical management in adult outpatients with DSM-III-R major depressive disorder with atypical features. Atypical features were operationalized as: reactive mood plus at least 2 of the following: hypersomnia, hyperphagia, leaden paralysis, or lifetime sensitivity to interpersonal rejection. The “double blind” was between phenelzine and matched pill placebo. The primary outcome was the 21-item Hamilton Rating Scale for Depression administrated by an independent evaluator blind to randomized cell assignment. This design choice controlled for patient and treating clinician expectancies, which may have been present at the time of blinded evaluation. It is accurate that to date the field has not identified a method to control for such expectancies at the time psychotherapy, of any type, is provided or received. Solutions are welcome. Response rates in an intention to treat sample (N=108) did not differ between phenelzine (58%) and cognitive therapy (58%); responses rates of both active treatments were significantly higher than in pill placebo (28%). These (early) findings are compatible with recent meta-analyses (e.g.,Weitz, Hollon, Twisk, et al., 2015).

On 2017 May 01, Doug Berger commented:

The title of this study is misleading because only the phenelzine groups were, and can be, double-blind. The cognitive therapy groups can not have blinded subjects (which would be a single-blind) nor blind treaters, they can only have masked raters (which should not be called single-blind either as single-blind is defined as only when subjects are blind which they are not in the cognitive therapy arms in this study). Additionally, masked raters only record the result of what comes out of the subject and treater system, so that emphasizing that raters are blind has little meaning on design rigor.

More importantly, comparing outcome in the unblinded cognitive therapy arms to the outcome of the double-blinded phenelzine arms is attempting to compare two groups of data born from a completely different design rigor. Subjects studied with subjective endpoints such as that seen in depression must be double-blinded in order to filter out any hope and expectation that subjects may have in these studies that have large random error and are prone to subjective bias.

See deeper discussion here: Double-Blinding and Bias in Medication and Cognitive-Behavioral Therapy Trials for Major Depressive Disorder. By Berger D. https://www.ncbi.nlm.nih.gov/pubmed/26870318.2

Douglas Berger, M.D., Ph.D. U.S. Board-Certified Psychiatrist Tokyo, Japan

On 2017 May 01, Doug Berger commented:

The title of this study is misleading because only the phenelzine groups were, and can be, double-blind. The cognitive therapy groups can not have blinded subjects (which would be a single-blind) nor blind treaters, they can only have masked raters (which should not be called single-blind either as single-blind is defined as only when subjects are blind which they are not in the cognitive therapy arms in this study). Additionally, masked raters only record the result of what comes out of the subject and treater system, so that emphasizing that raters are blind has little meaning on design rigor.

More importantly, comparing outcome in the unblinded cognitive therapy arms to the outcome of the double-blinded phenelzine arms is attempting to compare two groups of data born from a completely different design rigor. Subjects studied with subjective endpoints such as that seen in depression must be double-blinded in order to filter out any hope and expectation that subjects may have in these studies that have large random error and are prone to subjective bias.

See deeper discussion here: Double-Blinding and Bias in Medication and Cognitive-Behavioral Therapy Trials for Major Depressive Disorder. By Berger D. https://www.ncbi.nlm.nih.gov/pubmed/26870318.2

Douglas Berger, M.D., Ph.D. U.S. Board-Certified Psychiatrist Tokyo, Japan

On 2017 May 03, Robin B. Jarrett, PhD commented:

This randomized controlled trial evaluated the extent to which 10-weeks of acute phase phenelzine plus clinical management or cognitive therapy reduced depressive symptoms compared to pill placebo plus clinical management in adult outpatients with DSM-III-R major depressive disorder with atypical features. Atypical features were operationalized as: reactive mood plus at least 2 of the following: hypersomnia, hyperphagia, leaden paralysis, or lifetime sensitivity to interpersonal rejection. The “double blind” was between phenelzine and matched pill placebo. The primary outcome was the 21-item Hamilton Rating Scale for Depression administrated by an independent evaluator blind to randomized cell assignment. This design choice controlled for patient and treating clinician expectancies, which may have been present at the time of blinded evaluation. It is accurate that to date the field has not identified a method to control for such expectancies at the time psychotherapy, of any type, is provided or received. Solutions are welcome. Response rates in an intention to treat sample (N=108) did not differ between phenelzine (58%) and cognitive therapy (58%); responses rates of both active treatments were significantly higher than in pill placebo (28%). These (early) findings are compatible with recent meta-analyses (e.g.,Weitz, Hollon, Twisk, et al., 2015).

On 2017 May 03, Doug Berger commented:

MISTAKEN TITLE: THE COGNITIVE THERAPY GROUPS WERE NOT BLINDED & HAD NO BLIND-PLACEBO ARM

I thank Dr. Jarrett for replying, however, the issue of the misleading title is not resolved because the cognitive therapy arms are neither single- nor double-blinded, and neither is there a blind-placebo control for the cognitive therapy arms. Thus, it is a violation of clinical trial logic to compare the double-blinded medication arms to the unblind psychotherapy arms in this trial as the data is born from completely different designs. The authors would be well-respected to admit this serious infraction of trial logic and retract the paper (perhaps they were not well aware of this problem 20 years ago).

It is impossible to filter-out the effects of hope and expectation that may bias results in a condition such as depression that has subjective endpoints and large random error. Non-blind outcome research is only excusable in conditions with objectively measurable biological endpoints such as bone fracture rates, myocardial infarction, stroke, tumor size etc.

It is known that 30% of patients receiving drug or psychotherapy placebo can be deemed “responder” in depression. Even a small amount of bias can add up when a study is unblinded as in a psychotherapy so that a large N can make a biased study seem significant. Jarrett has fallen on the slippery slope of opining that blind-raters can make up for some kind of bias in the subject-treater system (unblind treaters also impart hope and expectation to the patient), raters only rate what the subjects’ report.

Antidepressants on the market were required to prove efficacy in double-blind conditions against blind-placebo per FDA regulation-there is no agency that directs the rigor of a psychotherapy outcome trial. The hurdle to do this (a superiority study) is much higher than the non-inferiority results that Jarrett notes as “did not differ”. Comparing blind medication and unblind psychotherapy in the same study severely handicaps the medication arms and thus invalidates clinical trial logic.

If one wishes to look at meta analyses, this one here found only small treatment effects for cognitive behavioural therapy in Major Depression when controlled for blinding: Lynch D, Laws KR, McKenna PJ: Cognitive behavioural therapy for major psychiatric disorder: does it really work? A meta-analytical review of well-controlled trials. Psychol Med. 2010; 40(1): 9–24.

Regardless of quoting meta analyses, there is no way to get around the issue of the inability to blind or have a blind-placebo comparator and thus the inability to filter out hope and expectation in subjects with subjective conditions such as a depressive disorder. The inability to blind psychotherapy trials is part of a larger problem in cognitive behavioural therapy, derivatives such as DBT, as well as other psychotherapy outcome research. Unfortunately both professionals and the lay public alike mistakenly believe that "randomized" and "blind raters" indicates rigorous study for depression (and other psychiatric conditions) as they are not aware of the lack of double-blinding.

The other points in this post are all repeated and referenced in the article noted below: https://www.ncbi.nlm.nih.gov/pubmed/26870318.2