On 2017 Jan 11, Tania M O Abe commented:

After reviewing the entire paper, we noticed an error in data of the last column of Table 2. During the registration of information in Table 2, the last column mistakenly recorded incorrect monthly number of deaths for myocardial infarction. The correction will be done this week. Once this is an government data, it can be found in http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sim/cnv/obt10SP.def

On 2016 Dec 27, Clive Bates commented:

The reporting of this study is highly misleading. The abstract confidently asserts:

RESULTS: We observed a reduction in mortality rate (-11.9% in the first 17 months after the law) and in hospital admission rate (-5.4% in the first 3 months after the law) for myocardial infarction after the implementation of the smoking ban law.

In fact, those with access to the full study will not find these observed reductions anywhere. Both myocardial infarction (heart attack) deaths and hospital admissions increased substantially after the smoking ban came into effect in August 2009. See graphs of the study data for myocardial infarction deaths and hospital admissions courtesy of Chris Snowden's blog post on these findings: Brazilian Smoking Ban Miracle.

The supposed 'decrease' emerges from modelling that adjusts for other factors that may influence heart attacks to create a counterfactual (what would have been expected to happen without the smoking ban). These factors give predicted rates of heart attack deaths and hospital admissions over the period studied. But why would the predicted rates suddenly shoot up to levels unprecedented in the dataset and so high that the observed large observed increases represent a decline compared to the even-higher prediction?

The explanation given in the paper is as follows:

The Autoregressive Integrated Moving Average with exogenous variables (ARIMAX) method was used to analyse the effect of the smoking ban law, modelled as a dummy variable, in the mortality rate and hospital admission rate data for myocardial infarction. The ARIMAX models were also adjusted to other parameters, including ‘total hospital admission’, CO, minimum temperature and air relative humidity. The ARIMAX method allows to estimate lag effects of input series and to forecast output series, as a function of a linear filter of the input series (transfer function) and of the noise (ARIMA filter) and by controlling for the autocorrelations. It enables us to compare the predicted rate of hospital admission and mortality with the real observed rate.

But how these corrections are made and whether they are valid is barely justified in the paper - they are buried in the black-box model used and reported uncritically. Given that these adjustments reverse the observed effects - turning a sharp rise into a decline - then surely the authors should have asked themselves harder questions and not just trusted the model and their choice of inputs to it. However, they don't even remark on this change of the sign of the effect in the paper as if the actual observations are an embarrassment to be ignored rather than discussed. Yet this is the most striking feature of the paper. Had the authors wished to explain their work transparently, they could have plotted the counterfactual (the predicted values for deaths and admissions with no ban) and the actual emissions and shown the decrease that way. But that would have begged the question: what is causing the very steep predicted rise? Or raised the possibility of modelling error or rogue assumptions.

Surely, confronted with this highly counterintuitive result, the editors and peer-reviewers should have demanded more explanation for the choice of confounding variables, a sensitivity analysis to flex whatever opaque assumptions have been made, publication of the data used to make adjustments, and a plausible narrative to explain the reversal of an increase to a decrease and the implicit massive underlying increase in background hospital admission and MI mortality rate that apparently coincided with the smoking ban. Finally, whatever the methodology, it is highly misleading to report these adjusted figures as an observed reduction in the abstract, especially with the faux precision of one decimal point.

I would like to suggest the following rewording of the results for inclusion a revised abstract:

RESULTS: We observed a substantial increase in mortality and hospital admissions for myocardial infarction after the implementation of the smoking ban law in Sao Paulo in August 2009. However, it is possible that other factors are responsible for this increase. After hand-picking a small number of possible confounding variables, and applying opaque statistical adjustments to account for their effect though without providing the data necessary for verification, we have been able to demonstrate that these increases could represent a modelled reduction in mortality attributable the smoking ban (−11.9% in the first 17 months after the law) and in hospital admission rate (−5.4% in the first 3 months after the law).

CONCLUSIONS: Hospital admissions and mortality rate for myocardial infarction were increased in the first months after the comprehensive smoking ban law was implemented. However, it is possible that factors other than the smoking ban accounted for some or all of this.

One must be concerned about the role of the journal Tobacco Control. Is this journal really an easy conduit for admitting studies of dubious quality to the peer-reviewed literature simply because the findings appear to provide support for certain tobacco control policies? I would welcome the editors' comments as well as that of the authors.

Please see original commentary from Dr Michael Siegel, Professor in the Department of Community Health Sciences, Boston University School of Public Health on his blog here and here.

On 2016 Dec 27, Clive Bates commented:

The reporting of this study is highly misleading. The abstract confidently asserts:

RESULTS: We observed a reduction in mortality rate (-11.9% in the first 17 months after the law) and in hospital admission rate (-5.4% in the first 3 months after the law) for myocardial infarction after the implementation of the smoking ban law.

In fact, those with access to the full study will not find these observed reductions anywhere. Both myocardial infarction (heart attack) deaths and hospital admissions increased substantially after the smoking ban came into effect in August 2009. See graphs of the study data for myocardial infarction deaths and hospital admissions courtesy of Chris Snowden's blog post on these findings: Brazilian Smoking Ban Miracle.

The supposed 'decrease' emerges from modelling that adjusts for other factors that may influence heart attacks to create a counterfactual (what would have been expected to happen without the smoking ban). These factors give predicted rates of heart attack deaths and hospital admissions over the period studied. But why would the predicted rates suddenly shoot up to levels unprecedented in the dataset and so high that the observed large observed increases represent a decline compared to the even-higher prediction?

The explanation given in the paper is as follows:

The Autoregressive Integrated Moving Average with exogenous variables (ARIMAX) method was used to analyse the effect of the smoking ban law, modelled as a dummy variable, in the mortality rate and hospital admission rate data for myocardial infarction. The ARIMAX models were also adjusted to other parameters, including ‘total hospital admission’, CO, minimum temperature and air relative humidity. The ARIMAX method allows to estimate lag effects of input series and to forecast output series, as a function of a linear filter of the input series (transfer function) and of the noise (ARIMA filter) and by controlling for the autocorrelations. It enables us to compare the predicted rate of hospital admission and mortality with the real observed rate.

But how these corrections are made and whether they are valid is barely justified in the paper - they are buried in the black-box model used and reported uncritically. Given that these adjustments reverse the observed effects - turning a sharp rise into a decline - then surely the authors should have asked themselves harder questions and not just trusted the model and their choice of inputs to it. However, they don't even remark on this change of the sign of the effect in the paper as if the actual observations are an embarrassment to be ignored rather than discussed. Yet this is the most striking feature of the paper. Had the authors wished to explain their work transparently, they could have plotted the counterfactual (the predicted values for deaths and admissions with no ban) and the actual emissions and shown the decrease that way. But that would have begged the question: what is causing the very steep predicted rise? Or raised the possibility of modelling error or rogue assumptions.

Surely, confronted with this highly counterintuitive result, the editors and peer-reviewers should have demanded more explanation for the choice of confounding variables, a sensitivity analysis to flex whatever opaque assumptions have been made, publication of the data used to make adjustments, and a plausible narrative to explain the reversal of an increase to a decrease and the implicit massive underlying increase in background hospital admission and MI mortality rate that apparently coincided with the smoking ban. Finally, whatever the methodology, it is highly misleading to report these adjusted figures as an observed reduction in the abstract, especially with the faux precision of one decimal point.

I would like to suggest the following rewording of the results for inclusion a revised abstract:

RESULTS: We observed a substantial increase in mortality and hospital admissions for myocardial infarction after the implementation of the smoking ban law in Sao Paulo in August 2009. However, it is possible that other factors are responsible for this increase. After hand-picking a small number of possible confounding variables, and applying opaque statistical adjustments to account for their effect though without providing the data necessary for verification, we have been able to demonstrate that these increases could represent a modelled reduction in mortality attributable the smoking ban (−11.9% in the first 17 months after the law) and in hospital admission rate (−5.4% in the first 3 months after the law).

CONCLUSIONS: Hospital admissions and mortality rate for myocardial infarction were increased in the first months after the comprehensive smoking ban law was implemented. However, it is possible that factors other than the smoking ban accounted for some or all of this.

One must be concerned about the role of the journal Tobacco Control. Is this journal really an easy conduit for admitting studies of dubious quality to the peer-reviewed literature simply because the findings appear to provide support for certain tobacco control policies? I would welcome the editors' comments as well as that of the authors.

Please see original commentary from Dr Michael Siegel, Professor in the Department of Community Health Sciences, Boston University School of Public Health on his blog here and here.

On 2017 Jan 11, Tania M O Abe commented:

After reviewing the entire paper, we noticed an error in data of the last column of Table 2. During the registration of information in Table 2, the last column mistakenly recorded incorrect monthly number of deaths for myocardial infarction. The correction will be done this week. Once this is an government data, it can be found in http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sim/cnv/obt10SP.def