On 2014 Feb 04, Jacob Puliyel commented:

This comment has been written after correspondence through the journal with the authors.

The WHO considers programmes cost-effective, if they cost less than 3 X GDP per capita per DALY averted. Such thresholds are less meaningful in developing countries where funds are limited and there are numerous worthwhile projects (costing even less than 1 X GDP per capita per DALY), vying for scarce health care resources. Efficient allocation of resources involves selecting programmes that provide the best value for money till the budget is exhausted. The base-case cost-effectiveness is used for this selection of programmes. It is therefore important that this base-case-estimate must be as accurate as possible.

Clark and colleagues note that the addition of the Hib vaccine will increase the cost of immunization per child four fold in India. They found that incidence of Hib pneumonia deaths and efficacy of the vaccine against Hib pneumonia, were factors that crucially influenced cost effectiveness.

We find that the authors have used data that overestimates benefits of the vaccine.

Pneumonia

Evidence based medicine requires the use of the best evidence available for decision making and when properly conducted RCTs and case control data are available, other less empirical estimates of Hib disease burden should not be used.

a) The authors write that their estimate that 7% of all pneumonias are caused by Hib is broadly consistent with the pooled 5% reduction in radiological pneumonia found in combining results of 2 Hib vaccine studies in Asia. Gessner BD, 2005, Baqui AH, 2007 They use inverse variance meta analysis with 64% weight assigned to the Indonesian study. However they write, mistakenly, that the Indonesian study found vaccine efficacy of -10% (95% CI -33% to 9), where the study actually reported efficacy of -12% ( 95% CI -33 to 9). In the Bangladesh study vaccine efficacy was 16% (CI -11 to 37) for pneumonia identified by WHO experts (n=675) and not 32% as stated in the cost-effectiveness report.<br> Meta analysis combining the vaccine efficiency from this case control study in Bangladesh with the vaccine efficacy in the cohort study from Indonesia was not possible with the data published in the two papers. However the arithmetic mean using the weights provided by Clark et al (Indonesia (weight 64%): vaccine efficacy -12% and Bangladesh (weight 36%): vaccine efficacy 16%; it gives a pooled efficacy is -1.9% (and not 5% as reported in the paper).

b) The author did sensitivity testing assuming vaccine efficacy against pneumonia to be 50% of the base case figure used by the authors, which was 7%. In their worst case scenario, vaccine efficacy was 3.5% which is more optimistic than even the calculated base-case rate of vaccine efficacy of -1.9%.

c) They estimate that 7% of pneumonia deaths in children aged 1-59 months are caused by Hib, just because according to them, Hib is responsible for 7% of all pneumonias. This can be true only is all pneumonia-causing-bacteria are equally lethal. There is no evidence in literature to support this assumption.

Meningitis

In the same way the authors used data from the Minz study on Hib meningitis. Minz S, 2008 However they trebled the incidence to 22/100,000, ostensibly to account for children without access to hospitals and cases not detected in the laboratory. Inflating the meningitis numbers was not reasonable for the following reasons:

a. The Minz study was a community based with households visited fortnightly. Tests were done free of cost at the nearest hospital. Adjusting for poor access here was superfluous.

b. Minz used a Latex Agglutination Test (LAT) which is particularly sensitive for detecting Hib (Sensitivity 93 per cent for Hib infections but 39 per cent for Neisseria meningitides). LAT would miss only 7% cases of Hib. Most of cases of purulent meningitis where no organism was identified were probably not Hib disease.

c. The authors assume that parents, who don’t bring children to hospital when they have respiratory infections (a relatively trivial disorder), will not bring them when suffering from meningitis either. In doing so they overestimate the numbers not presenting to hospital with meningitis and in them they assume mortality would be 100%. The burden of Hib meningitis has been exaggerated in this way. The authors have written that even when they tested a scenario with the unadjusted Hib meningitis incidence (7 per 100,000 per year, <5yrs) as a minimal estimate, the cost per DALY averted remained below 3 GDP per capita. As discussed in the introduction such thresholds are not very useful for developing countries.

Non pneumonia Non meningitis

As the incidence of non pneumonia non meningitis (NPNM) is taken directly as a fraction of the incidence of meningitis; exaggerating the incidence of meningitis three fold (compared to empirical data from Minz) has the effect of trebling the calculated incidence of NPNM disease. The authors have responded that Hib NPNM accounts for only 1% of the estimated deaths prevented. Reducing the contribution of Hib NPNM would not alter the findings of the paper.

Conclusion

We hope the authors will be able to provide a more accurate base-case estimate of costs and benefits in the light of the above discussion. Such a base case estimate must include cost of treating the 1.9% increase in pneumonia in the vaccinated and also include the increased deaths from pneumonia.

Pinky Bahl MD, Supriya Rastogi DCH, Jacob Puliyel MD

On 2014 Feb 04, Jacob Puliyel commented:

This comment has been written after correspondence through the journal with the authors.

The WHO considers programmes cost-effective, if they cost less than 3 X GDP per capita per DALY averted. Such thresholds are less meaningful in developing countries where funds are limited and there are numerous worthwhile projects (costing even less than 1 X GDP per capita per DALY), vying for scarce health care resources. Efficient allocation of resources involves selecting programmes that provide the best value for money till the budget is exhausted. The base-case cost-effectiveness is used for this selection of programmes. It is therefore important that this base-case-estimate must be as accurate as possible.

Clark and colleagues note that the addition of the Hib vaccine will increase the cost of immunization per child four fold in India. They found that incidence of Hib pneumonia deaths and efficacy of the vaccine against Hib pneumonia, were factors that crucially influenced cost effectiveness.

We find that the authors have used data that overestimates benefits of the vaccine.

Pneumonia

Evidence based medicine requires the use of the best evidence available for decision making and when properly conducted RCTs and case control data are available, other less empirical estimates of Hib disease burden should not be used.

a) The authors write that their estimate that 7% of all pneumonias are caused by Hib is broadly consistent with the pooled 5% reduction in radiological pneumonia found in combining results of 2 Hib vaccine studies in Asia. Gessner BD, 2005, Baqui AH, 2007 They use inverse variance meta analysis with 64% weight assigned to the Indonesian study. However they write, mistakenly, that the Indonesian study found vaccine efficacy of -10% (95% CI -33% to 9), where the study actually reported efficacy of -12% ( 95% CI -33 to 9). In the Bangladesh study vaccine efficacy was 16% (CI -11 to 37) for pneumonia identified by WHO experts (n=675) and not 32% as stated in the cost-effectiveness report.<br> Meta analysis combining the vaccine efficiency from this case control study in Bangladesh with the vaccine efficacy in the cohort study from Indonesia was not possible with the data published in the two papers. However the arithmetic mean using the weights provided by Clark et al (Indonesia (weight 64%): vaccine efficacy -12% and Bangladesh (weight 36%): vaccine efficacy 16%; it gives a pooled efficacy is -1.9% (and not 5% as reported in the paper).

b) The author did sensitivity testing assuming vaccine efficacy against pneumonia to be 50% of the base case figure used by the authors, which was 7%. In their worst case scenario, vaccine efficacy was 3.5% which is more optimistic than even the calculated base-case rate of vaccine efficacy of -1.9%.

c) They estimate that 7% of pneumonia deaths in children aged 1-59 months are caused by Hib, just because according to them, Hib is responsible for 7% of all pneumonias. This can be true only is all pneumonia-causing-bacteria are equally lethal. There is no evidence in literature to support this assumption.

Meningitis

In the same way the authors used data from the Minz study on Hib meningitis. Minz S, 2008 However they trebled the incidence to 22/100,000, ostensibly to account for children without access to hospitals and cases not detected in the laboratory. Inflating the meningitis numbers was not reasonable for the following reasons:

a. The Minz study was a community based with households visited fortnightly. Tests were done free of cost at the nearest hospital. Adjusting for poor access here was superfluous.

b. Minz used a Latex Agglutination Test (LAT) which is particularly sensitive for detecting Hib (Sensitivity 93 per cent for Hib infections but 39 per cent for Neisseria meningitides). LAT would miss only 7% cases of Hib. Most of cases of purulent meningitis where no organism was identified were probably not Hib disease.

c. The authors assume that parents, who don’t bring children to hospital when they have respiratory infections (a relatively trivial disorder), will not bring them when suffering from meningitis either. In doing so they overestimate the numbers not presenting to hospital with meningitis and in them they assume mortality would be 100%. The burden of Hib meningitis has been exaggerated in this way. The authors have written that even when they tested a scenario with the unadjusted Hib meningitis incidence (7 per 100,000 per year, <5yrs) as a minimal estimate, the cost per DALY averted remained below 3 GDP per capita. As discussed in the introduction such thresholds are not very useful for developing countries.

Non pneumonia Non meningitis

As the incidence of non pneumonia non meningitis (NPNM) is taken directly as a fraction of the incidence of meningitis; exaggerating the incidence of meningitis three fold (compared to empirical data from Minz) has the effect of trebling the calculated incidence of NPNM disease. The authors have responded that Hib NPNM accounts for only 1% of the estimated deaths prevented. Reducing the contribution of Hib NPNM would not alter the findings of the paper.

Conclusion

We hope the authors will be able to provide a more accurate base-case estimate of costs and benefits in the light of the above discussion. Such a base case estimate must include cost of treating the 1.9% increase in pneumonia in the vaccinated and also include the increased deaths from pneumonia.

Pinky Bahl MD, Supriya Rastogi DCH, Jacob Puliyel MD