On 2016 May 02, Ricky Turgeon commented:

This review required withdrawal for additional important reasons to the outdated search described by the editors, including inappropriate study inclusion, factually incorrect statements and conclusions not supported by the included evidence. As these reasons were not described in the reasons for withdrawal, we share a letter sent to the Cochrane Heart Group on March 30, prior to withdrawal, highlighting these issues:

"Dear Cochrane Heart Group,

In performing our own review and synthesis of the evidence for impact of diuretics on mortality and hospitalization in patients with heart failure, we identified critical issues in Cochrane review CD003838 that warrant immediate withdrawal of the review for revision.1 For the purposes of this letter, we focus on the analyses of ‘mortality’ and ‘heart failure worsening’ and 3 of the 4 randomized controlled trials (RCTs) that provided data for at least one of these analyses.2-4

First, the reviewers defined eligible participants as “adult participants with chronic heart failure, […] a clinical syndrome characterised by breathlessness and fatigue that is caused by an inability of the heart to support an adequate circulation, that may limit exercise tolerance and may lead to pulmonary congestion and peripheral oedema”. Based on these criteria, reviewers should have excluded the trials by Burr, de Jonge, and Myers from this Cochrane review as none or few patients in these trials met this disease definition.1-3

In the trial by Burr et al,2 investigators excluded patients if “they had had congestive cardiac failure during the previous three months” or “they had ever had left ventricular failure”. The published report further notes that “an attempt was made to discover the original reason for which each patient had been given a diuretic” and that “in most cases, however, this information was not in the hospital notes”. They furthermore note that “ankle oedema was often mentioned in the notes, but in the majority there was no reference to cardiac failure”.

The trial by de Jonge et al specifically excluded patients with heart failure.3 First, the authors describe their objective as “to determine the effect of withdrawing diuretic drugs on oedema in patients prescribed them for only ankle oedema, excluding patients with cardiac […] failure”. In their methods, the authors describe their approach to excluding patients with a clear diagnosis of heart failure, such as those “[having] congestive heart failure or increased risk of developing it after stopping diuretic drugs,” or “heart failure previously established by a cardiologist, history of severe dyspnoea treated by the general practitioner as cardiac failure, atrial fibrillation, symptoms of right sided heart failure, palpable right ventricular pulsations, or hepatomegaly”. Therefore, de Jonge et al appeared to include only patients in whom heart failure was unlikely or ruled-out as the cause of ankle edema. Additionally, the Cochrane authors describe this trial as only including “participants with decreased ejection fraction (EF) measured by echocardiography”, but this is not described in the published report of the de Jonge trial.1,3

Similarly, the eligibility criteria in the trial by Myers et al excluded patients with definite or probable heart failure.4 In this trial, “concurrent digoxin therapy” was the most common reason for exclusion, with additional relevant reasons for exclusion being “active heart failure […] (clinical or radiological evidence of heart failure)” or hypertension. Corroborating this is the fact that only 9 out of 77 included patients were noted to have had “previous CHF” according to the study report’s patient characteristics table.

Based on the above published trial details, it is clear that 3 of the 4 studies contributing data to the Cochrane review’s ‘mortality’ and ‘worsening heart failure’ outcome analyses should be excluded according to the reviewers’ predefined review eligibility criteria.1-4 Exclusion of the Burr and Myers trials leaves only the Sherman trial for the ‘mortality’ analysis, demonstrating no statistically significant difference between diuretics and control (0 versus 2 deaths).1 Additionally, no trials remain for analysis for the ‘heart failure worsening’ outcome with exclusion of the Burr and de Jonge trials. As a result, exclusion of trials not meeting this Cochrane review’s predefined eligibility criteria substantially changes the review’s conclusions.

Second, further issues arise from attributing a causal role of diuretics in the reported reduction in mortality. In the trial by Burr et al, none of the 3 deaths in the control group were attributable to heart failure.2 Similarly, only 1 of the 8 deaths reported in the control group of the trial by Myers et al was attributable to heart failure, with the others attributed to cancer (3), respiratory disease (2), stroke (1) or gastrointestinal bleed (1).4 Such inconsistencies between all-cause mortality and heart failure-related mortality would deserve, at minimum, description by the reviewers in their Discussion section. Ideally, the impact of inconsistency between outcomes should be considered in the determination of quality of the body of evidence, such as by using the framework provided by the GRADE approach as described in the Cochrane Handbook.5

Third, the reviewers inappropriately exclude the results of the Myers trial from their analysis of ‘worsening of heart failure’.1 According to the authors, they excluded Myers from this analysis “because of heterogeneity for heart failure worsening in the diuretic group versus placebo (chi-square, 16.03; P = 0.001)”. The heterogeneity noted resulted from the increase in “withdrawal due to heart failure” in the diuretic group compared to the placebo group in this trial (6/29 versus 2/29). Such arbitrary exclusion from analysis constitutes selective outcome reporting bias. Rather than excluding the trial by Myers et al, which contributed greater statistical weight (38.9%) than either trials by Burr or de Jonge, the appropriate course of action according to the Cochrane Handbook would have been to evaluate methodological and clinical sources of heterogeneity, and to abstain from performing a meta-analysis of this outcome.5

Based on the above appraisal of critical issues present in Cochrane review CD003838, we urge editors of the Cochrane Heart Group to withdraw the aforementioned review from the Cochrane Library and issue a report on the Cochrane Heart Group website describing reasons for withdrawal. Authors of the review should then be provided with the opportunity to revise the review to meet the standards set by their protocol.

Thank you for your consideration,

Ricky Turgeon BScPharm, ACPR, PharmD Michael Kolber BSc, MD, CCFP, MSc

References 1. Faris RF, Flather M, Purcell H, Poole-Wilson PA, Coats AJS. Diuretics for heart failure. Cochrane Database Syst Rev 2012;2:CD003838. 2. Burr ML, King S, Davies HE, Pathy MS. The effects of discontinuing long-term diuretic therapy in the elderly. Age Ageing 1977;6:38-45. 3. de Jonge JW, Knottnerus JA, van Zutphen WM, de Bruijne GA, Struijker Boudier HA. Short term effect of withdrawal of diuretic drugs prescribed for ankle oedema. BMJ 1994;308:511-3. 4. Myers MG, Weingert ME, Fisher RH, Gryfe CI, Shulman HS. Unnecessary diuretic therapy in the elderly. Age Ageing 1982;11:213-21. 5. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org."

On 2016 May 02, Ricky Turgeon commented:

This review required withdrawal for additional important reasons to the outdated search described by the editors, including inappropriate study inclusion, factually incorrect statements and conclusions not supported by the included evidence. As these reasons were not described in the reasons for withdrawal, we share a letter sent to the Cochrane Heart Group on March 30, prior to withdrawal, highlighting these issues:

"Dear Cochrane Heart Group,

In performing our own review and synthesis of the evidence for impact of diuretics on mortality and hospitalization in patients with heart failure, we identified critical issues in Cochrane review CD003838 that warrant immediate withdrawal of the review for revision.1 For the purposes of this letter, we focus on the analyses of ‘mortality’ and ‘heart failure worsening’ and 3 of the 4 randomized controlled trials (RCTs) that provided data for at least one of these analyses.2-4

First, the reviewers defined eligible participants as “adult participants with chronic heart failure, […] a clinical syndrome characterised by breathlessness and fatigue that is caused by an inability of the heart to support an adequate circulation, that may limit exercise tolerance and may lead to pulmonary congestion and peripheral oedema”. Based on these criteria, reviewers should have excluded the trials by Burr, de Jonge, and Myers from this Cochrane review as none or few patients in these trials met this disease definition.1-3

In the trial by Burr et al,2 investigators excluded patients if “they had had congestive cardiac failure during the previous three months” or “they had ever had left ventricular failure”. The published report further notes that “an attempt was made to discover the original reason for which each patient had been given a diuretic” and that “in most cases, however, this information was not in the hospital notes”. They furthermore note that “ankle oedema was often mentioned in the notes, but in the majority there was no reference to cardiac failure”.

The trial by de Jonge et al specifically excluded patients with heart failure.3 First, the authors describe their objective as “to determine the effect of withdrawing diuretic drugs on oedema in patients prescribed them for only ankle oedema, excluding patients with cardiac […] failure”. In their methods, the authors describe their approach to excluding patients with a clear diagnosis of heart failure, such as those “[having] congestive heart failure or increased risk of developing it after stopping diuretic drugs,” or “heart failure previously established by a cardiologist, history of severe dyspnoea treated by the general practitioner as cardiac failure, atrial fibrillation, symptoms of right sided heart failure, palpable right ventricular pulsations, or hepatomegaly”. Therefore, de Jonge et al appeared to include only patients in whom heart failure was unlikely or ruled-out as the cause of ankle edema. Additionally, the Cochrane authors describe this trial as only including “participants with decreased ejection fraction (EF) measured by echocardiography”, but this is not described in the published report of the de Jonge trial.1,3

Similarly, the eligibility criteria in the trial by Myers et al excluded patients with definite or probable heart failure.4 In this trial, “concurrent digoxin therapy” was the most common reason for exclusion, with additional relevant reasons for exclusion being “active heart failure […] (clinical or radiological evidence of heart failure)” or hypertension. Corroborating this is the fact that only 9 out of 77 included patients were noted to have had “previous CHF” according to the study report’s patient characteristics table.

Based on the above published trial details, it is clear that 3 of the 4 studies contributing data to the Cochrane review’s ‘mortality’ and ‘worsening heart failure’ outcome analyses should be excluded according to the reviewers’ predefined review eligibility criteria.1-4 Exclusion of the Burr and Myers trials leaves only the Sherman trial for the ‘mortality’ analysis, demonstrating no statistically significant difference between diuretics and control (0 versus 2 deaths).1 Additionally, no trials remain for analysis for the ‘heart failure worsening’ outcome with exclusion of the Burr and de Jonge trials. As a result, exclusion of trials not meeting this Cochrane review’s predefined eligibility criteria substantially changes the review’s conclusions.

Second, further issues arise from attributing a causal role of diuretics in the reported reduction in mortality. In the trial by Burr et al, none of the 3 deaths in the control group were attributable to heart failure.2 Similarly, only 1 of the 8 deaths reported in the control group of the trial by Myers et al was attributable to heart failure, with the others attributed to cancer (3), respiratory disease (2), stroke (1) or gastrointestinal bleed (1).4 Such inconsistencies between all-cause mortality and heart failure-related mortality would deserve, at minimum, description by the reviewers in their Discussion section. Ideally, the impact of inconsistency between outcomes should be considered in the determination of quality of the body of evidence, such as by using the framework provided by the GRADE approach as described in the Cochrane Handbook.5

Third, the reviewers inappropriately exclude the results of the Myers trial from their analysis of ‘worsening of heart failure’.1 According to the authors, they excluded Myers from this analysis “because of heterogeneity for heart failure worsening in the diuretic group versus placebo (chi-square, 16.03; P = 0.001)”. The heterogeneity noted resulted from the increase in “withdrawal due to heart failure” in the diuretic group compared to the placebo group in this trial (6/29 versus 2/29). Such arbitrary exclusion from analysis constitutes selective outcome reporting bias. Rather than excluding the trial by Myers et al, which contributed greater statistical weight (38.9%) than either trials by Burr or de Jonge, the appropriate course of action according to the Cochrane Handbook would have been to evaluate methodological and clinical sources of heterogeneity, and to abstain from performing a meta-analysis of this outcome.5

Based on the above appraisal of critical issues present in Cochrane review CD003838, we urge editors of the Cochrane Heart Group to withdraw the aforementioned review from the Cochrane Library and issue a report on the Cochrane Heart Group website describing reasons for withdrawal. Authors of the review should then be provided with the opportunity to revise the review to meet the standards set by their protocol.

Thank you for your consideration,

Ricky Turgeon BScPharm, ACPR, PharmD Michael Kolber BSc, MD, CCFP, MSc

References 1. Faris RF, Flather M, Purcell H, Poole-Wilson PA, Coats AJS. Diuretics for heart failure. Cochrane Database Syst Rev 2012;2:CD003838. 2. Burr ML, King S, Davies HE, Pathy MS. The effects of discontinuing long-term diuretic therapy in the elderly. Age Ageing 1977;6:38-45. 3. de Jonge JW, Knottnerus JA, van Zutphen WM, de Bruijne GA, Struijker Boudier HA. Short term effect of withdrawal of diuretic drugs prescribed for ankle oedema. BMJ 1994;308:511-3. 4. Myers MG, Weingert ME, Fisher RH, Gryfe CI, Shulman HS. Unnecessary diuretic therapy in the elderly. Age Ageing 1982;11:213-21. 5. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org."