On 2017 Nov 03, Elisabeth Schramm commented:

In reply to a comment by Falk Leichsenring

Allegiance effects controlled

Elisabeth Schramm, PhD; Levente Kriston, PhD; Ingo Zobel, PhD; Josef Bailer, PhD; Katrin Wambach, PhD; Matthias Backenstrass, PhD; Jan Philipp Klein, MD; Dieter Schoepf, MD; Knut Schnell, MD; Antje Gumz, MD; Paul Bausch, MSc; Thomas Fangmeier, PhD; Ramona Meister, MSc; Mathias Berger, MD; Martin Hautzinger, PhD; Martin Härter,MD, PhD

Corresponding Author: Elisabeth Schramm, PhD, Department of Psychiatry, Faculty of Medicine, University of Freiburg, Hauptstrasse 5, 79104 Freiburg, Germany (elisabeth.schramm@uniklinik-freiburg.de)

We acknowledge the comment of Drs. Steinert and Leichsenring (1) on our study (2) reasoning that our findings may at least in part be attributed to allegiance effects. Unfortunately, they provide neither a clarification of what they exactly refer to with the term “allegiance effects” nor a specific description of the presumed mechanisms (chain of effects) through which they think allegiance may have influenced our results. In fact, as specified both in the trial protocol (3) and the study report (2), we took a series of carefully safeguarded measures to minimize bias. Unlike stated in the comment, training and supervision of the study therapists and the center supervisors were performed by qualified and renowned experts for both investigated approaches (Martin Hautzinger for Supportive Psychotherapy and Elisabeth Schramm for Cognitive Behavioral Analysis System of Psychotherapy). Moreover, none of them has been involved in treating any study patients in this trial. We are confident that any possible allegiance of the participating researchers, therapists, supervisors, or other involved staff towards any, both, or none of the investigated interventions is very unlikely to have been able to surmount all of the implemented measures against bias and to affect the results substantially.

References

(1) Steinert C, Leichsenring F. The need to control for allegiance effects in psychotherapy research. PubMed Commons. Sep 08 2017

(2) Schramm E, Kriston L, Zobel I, Bailer J, Wambach K, Backenstrass M, Klein JP, Schoepf D, Schnell K, Gumz A, Bausch P, Fangmeier T, Meister R, Berger M, Hautzinger M, Härter M. Effect of Disorder-Specific vs Nonspecific Psychotherapy for Chronic Depression: A Randomized Clinical Trial. JAMA Psychiatry. Mar 01 2017; 74(3): 233-242

(3) Schramm E, Hautzinger M, Zobel I, Kriston L, Berger M, Härter M. Comparative efficacy of the Cognitive Behavioral Analysis System of Psychotherapy versus supportive psychotherapy for early onset chronic depression: design and rationale of a multisite randomized controlled trial. BMC Psychiatry. 2011;11:134

On 2017 Sep 08, Falk Leichsenring commented:

The need to control for allegiance effects in psychotherapy research

Christiane Steinert1,2, PhD and Falk Leichsenring, DSc2

1 Medical School Berlin, Department of Psychology, Calandrellistraße 1-9, 12247 Berlin

2 University of Giessen, Department of Psychosomatics and Psychotherapy, Ludwigstraße 76, 35392 Giessen

Corresponding author: Prof. Dr. Falk Leichsenring University of Giessen Department of Psychosomatics and Psychotherapy Ludwigstr. 76, 35392 Giessen, Germany Fon | +49-641-99 45647 Fax | +49-641-99 45669 Mail | falk-leichsenring@psycho.med.uni-giessen.de

In a recent trial on psychotherapeutic efficacy Schramm et al. hypothesized that Cognitive Behavioral Analysis System of Psychotherapy (CBASP) would be superior to supportive therapy (SP) in the treatment of chronic depression.1 This hypothesis was corroborated, CBASP improved depression significantly more, but only by a moderate effect size of 0.31. Some issues, however, raise the question of possible allegiance effects on several levels.2 (1) The authors clearly are in favour of CBASP and expect it to be superior to SP (primary hypothesis). (2) Furthermore, six authors participated as therapists in the CBASP group, while none of the authors seems to have participated in the SP group. The authors can be expected to be alleged to CBASP. (3) In addition, the therapy sessions were supervised by the very same authors participating in the CBASP group as therapists. (4) No expert of SP was listed to have participated in the study, neither as a researcher, a therapist nor as a supervisor. (5) The authors stated that all therapists completed a 3-year psychotherapeutic training program or were in an advanced stage of training. However, at least in Germany, there is no 3-year training program specifically for SP, only for CBT. Thus, it is unlikely that the therapists in the SP condition were alleged to SP in the same way as the therapists in the CBASP condition. Further information on the background of the therapists in SP would be informative.

Thus, a researcher, a therapist and a supervisor allegiance effect can be expected to be present in this study.3

Munder et al. found an association between researcher allegiance and outcome of r=0.35 which corresponds to a medium effect size.2 For this reason the possibility cannot be ruled out that the moderate between-group effect size of 0.31 is at least in part due to allegiance effects. - The fact that the treatments do not seem to differ with regard to treatment fidelity ratings does not rule out this possibility. This is also true for the fact that therapists met the criteria for mastery of CBASP and SP before treating study patients.

References

1. Schramm E, Kriston L, Zobel I, Bailer J, Wambach K, et al. Effect of Disorder-Specific vs Nonspecific Psychotherapy for Chronic Depression: A Randomized Clinical Trial. JAMA Psychiatry. Mar 01 2017;74(3):233-242.
2. Munder T, Flückiger, C, Gerger, H, Wampold, BE, Barth, J. Is the Allegiance Effect an Epiphenomenon of True Efficacy Differences Between Treatments? A Meta-Analysis. J Couns Psychol. 2012(Epub ahead of print).
3. Steinert C, Munder T, Rabung S, Hoyer J, Leichsenring F. Psychodynamic Therapy: As Efficacious as Other Empirically Supported Treatments? A Meta-Analysis Testing Equivalence of Outcomes. Am J Psychiatry. May 25 2017:appiajp201717010057.
4. Cuijpers P, Huibers MJ, Furukawa TA. The Need for Research on Treatments of Chronic Depression. JAMA Psychiatry. Mar 01 2017;74(3):242-243.

On 2017 Sep 08, Falk Leichsenring commented:

The need to control for allegiance effects in psychotherapy research

Christiane Steinert1,2, PhD and Falk Leichsenring, DSc2

1 Medical School Berlin, Department of Psychology, Calandrellistraße 1-9, 12247 Berlin

2 University of Giessen, Department of Psychosomatics and Psychotherapy, Ludwigstraße 76, 35392 Giessen

Corresponding author: Prof. Dr. Falk Leichsenring University of Giessen Department of Psychosomatics and Psychotherapy Ludwigstr. 76, 35392 Giessen, Germany Fon | +49-641-99 45647 Fax | +49-641-99 45669 Mail | falk-leichsenring@psycho.med.uni-giessen.de

In a recent trial on psychotherapeutic efficacy Schramm et al. hypothesized that Cognitive Behavioral Analysis System of Psychotherapy (CBASP) would be superior to supportive therapy (SP) in the treatment of chronic depression.1 This hypothesis was corroborated, CBASP improved depression significantly more, but only by a moderate effect size of 0.31. Some issues, however, raise the question of possible allegiance effects on several levels.2 (1) The authors clearly are in favour of CBASP and expect it to be superior to SP (primary hypothesis). (2) Furthermore, six authors participated as therapists in the CBASP group, while none of the authors seems to have participated in the SP group. The authors can be expected to be alleged to CBASP. (3) In addition, the therapy sessions were supervised by the very same authors participating in the CBASP group as therapists. (4) No expert of SP was listed to have participated in the study, neither as a researcher, a therapist nor as a supervisor. (5) The authors stated that all therapists completed a 3-year psychotherapeutic training program or were in an advanced stage of training. However, at least in Germany, there is no 3-year training program specifically for SP, only for CBT. Thus, it is unlikely that the therapists in the SP condition were alleged to SP in the same way as the therapists in the CBASP condition. Further information on the background of the therapists in SP would be informative.

Thus, a researcher, a therapist and a supervisor allegiance effect can be expected to be present in this study.3

Munder et al. found an association between researcher allegiance and outcome of r=0.35 which corresponds to a medium effect size.2 For this reason the possibility cannot be ruled out that the moderate between-group effect size of 0.31 is at least in part due to allegiance effects. - The fact that the treatments do not seem to differ with regard to treatment fidelity ratings does not rule out this possibility. This is also true for the fact that therapists met the criteria for mastery of CBASP and SP before treating study patients.

References

1. Schramm E, Kriston L, Zobel I, Bailer J, Wambach K, et al. Effect of Disorder-Specific vs Nonspecific Psychotherapy for Chronic Depression: A Randomized Clinical Trial. JAMA Psychiatry. Mar 01 2017;74(3):233-242.
2. Munder T, Flückiger, C, Gerger, H, Wampold, BE, Barth, J. Is the Allegiance Effect an Epiphenomenon of True Efficacy Differences Between Treatments? A Meta-Analysis. J Couns Psychol. 2012(Epub ahead of print).
3. Steinert C, Munder T, Rabung S, Hoyer J, Leichsenring F. Psychodynamic Therapy: As Efficacious as Other Empirically Supported Treatments? A Meta-Analysis Testing Equivalence of Outcomes. Am J Psychiatry. May 25 2017:appiajp201717010057.
4. Cuijpers P, Huibers MJ, Furukawa TA. The Need for Research on Treatments of Chronic Depression. JAMA Psychiatry. Mar 01 2017;74(3):242-243.

On 2017 Nov 03, Elisabeth Schramm commented:

In reply to a comment by Falk Leichsenring

Allegiance effects controlled

Elisabeth Schramm, PhD; Levente Kriston, PhD; Ingo Zobel, PhD; Josef Bailer, PhD; Katrin Wambach, PhD; Matthias Backenstrass, PhD; Jan Philipp Klein, MD; Dieter Schoepf, MD; Knut Schnell, MD; Antje Gumz, MD; Paul Bausch, MSc; Thomas Fangmeier, PhD; Ramona Meister, MSc; Mathias Berger, MD; Martin Hautzinger, PhD; Martin Härter,MD, PhD

Corresponding Author: Elisabeth Schramm, PhD, Department of Psychiatry, Faculty of Medicine, University of Freiburg, Hauptstrasse 5, 79104 Freiburg, Germany (elisabeth.schramm@uniklinik-freiburg.de)

We acknowledge the comment of Drs. Steinert and Leichsenring (1) on our study (2) reasoning that our findings may at least in part be attributed to allegiance effects. Unfortunately, they provide neither a clarification of what they exactly refer to with the term “allegiance effects” nor a specific description of the presumed mechanisms (chain of effects) through which they think allegiance may have influenced our results. In fact, as specified both in the trial protocol (3) and the study report (2), we took a series of carefully safeguarded measures to minimize bias. Unlike stated in the comment, training and supervision of the study therapists and the center supervisors were performed by qualified and renowned experts for both investigated approaches (Martin Hautzinger for Supportive Psychotherapy and Elisabeth Schramm for Cognitive Behavioral Analysis System of Psychotherapy). Moreover, none of them has been involved in treating any study patients in this trial. We are confident that any possible allegiance of the participating researchers, therapists, supervisors, or other involved staff towards any, both, or none of the investigated interventions is very unlikely to have been able to surmount all of the implemented measures against bias and to affect the results substantially.

References

(1) Steinert C, Leichsenring F. The need to control for allegiance effects in psychotherapy research. PubMed Commons. Sep 08 2017

(2) Schramm E, Kriston L, Zobel I, Bailer J, Wambach K, Backenstrass M, Klein JP, Schoepf D, Schnell K, Gumz A, Bausch P, Fangmeier T, Meister R, Berger M, Hautzinger M, Härter M. Effect of Disorder-Specific vs Nonspecific Psychotherapy for Chronic Depression: A Randomized Clinical Trial. JAMA Psychiatry. Mar 01 2017; 74(3): 233-242

(3) Schramm E, Hautzinger M, Zobel I, Kriston L, Berger M, Härter M. Comparative efficacy of the Cognitive Behavioral Analysis System of Psychotherapy versus supportive psychotherapy for early onset chronic depression: design and rationale of a multisite randomized controlled trial. BMC Psychiatry. 2011;11:134