On 2017 Sep 18, Konstantinos Farsalinos commented:

Although Jensen et al. mentioned in the 2015 NEJM research letter that the health risks of formaldehyde hemiacetal inhalation are unknown ("How formaldehyde-releasing agents behave in the respiratory tract is unknown..."), they made a calculation that the formaldehyde-attributable cancer risk from e-cigarette use is 5 to 15 times higher than from long-term smoking. These two statements are clearly contradictory, and the calculation of any cancer risk from formaldehyde hemiacetal emissions is invalid since no such risk has been established for these compounds. It is simply based on an unsubstantiated assumption that "inhaling formaldehyde-releasing agents carries the same risk per unit of formaldehyde as the risk associated with inhaling gaseous formaldehyde...". Additionally, the latter statement was further misinterpreted by the media that reported the cancer risk of e-cigarettes (total risk, not formaldehyde-attributable risk) being 5 to 15 times higher than smoking. This is extremely important because, according to Fowles and Dybing (http://tobaccocontrol.bmj.com/content/12/4/424.long), the formaldehyde-attributable cancer risk is less than 1% of the total cancer risk of smoking. Prof Peyton, one of the authors of the NEJM research letter, now claims that formaldehyde hemiacetals are different from gaseous formaldehyde. But then, why did he and his co-authors use the hemiacetal measurements to present the cancer risk of e-cigarettes relative to tobacco cigarettes, considering that literature data for gaseous formaldehyde emissions from tobacco cigarettes were used in the comparison?

In the recently published paper (https://www.nature.com/articles/s41598-017-11499-0), Salamanca et al. are comparing the results of an experimental method (NMR) with the results of an established and validated analytical method used to identify and measure aldehyde emissions. In our study, using a validated and widely accepted method, we found 89% higher levels of formaldehyde at 5 V than the level of formaldehyde hemiacetal reported by Jensen et al. in the NEJM research letter. Therefore, we did not underestimate formaldehyde levels. We did not reject the findings by Jensen et al. that, high formaldehyde levels can be produced when blindly testing e-cigarettes at inappropriately high power settings relevant to the atomizer used (in fact we found more, but in general our findings support their observations). However, this cannot be used as an indication that e-cigarette users are exposed to such extreme formaldehyde levels in their daily routine.

I would also like to remind that we used a product which is outdated and inefficiently designed, a design that has been abandoned since 2012. These products are not even available in the market in the European Union anymore. Recent atomizers release formaldehyde at levels orders of magnitude lower than the one we tested. Even if we assume, despite the lack of evidence, that formaldehyde hemiacetals carry the same health risk as formaldehyde and that DNP derivatization underestimates the total formaldehyde exposure, still e-cigarettes developed over that last 3 years (at least) emit by far lower formaldehyde than tobacco cigarettes. Finally, the comparison between tobacco and electronic cigarettes in formaldehyde emissions is based on measurements performed using the same analytical method (DNPH derivatization) for both products. The assumption about underestimation of formaldehyde by DNPH derivatization applies equally to tobacco cigarettes and electronic cigarettes (tobacco cigarettes contain a lot of PG and VG); thus the relative difference remains the same.

Recent studies have found that e-cigarettes release formaldehyde at levels corresponding to > 1900 cigarettes consumed per day (e.g. https://www.ncbi.nlm.nih.gov/pubmed/27461870). All these studies, which have been accompanied by press statements and wide media coverage, need to be replicated under verified realistic conditions. Science is defined by replication, and we hope that journal editors will accept the challenge of publishing studies that reject findings previously reported in the same journal. In any case, all our replication studies will be published in peer-reviewed journals.

In conclusion, our purpose was to replicate an experiment using a specific e-cigarette device, atomizer and liquid under verified realistic conditions in order to verify or reject the conclusion that e-cigarettes carry 5 to 15 times the cancer risk of tobacco cigarettes. Our study clearly identified that this statement is false, and this was evident even when we used a very outdated and inefficient e-cigarette product. Therefore, our study was a valid replication experiment that presented the clinical relevance of previous findings.

On 2017 Sep 14, David H Peyton commented:

We appreciate the feedback on our manuscript, and respect the concerns of the posters here as well as those of the authors of the 2017 Food Chem Toxicol. paper by Farsalinos et al., cited by Clive Bates below. We share in the common goal of maximizing harm reduction. However, the claim that the aforementioned manuscript embodies a “replication” of our study in NEJM is false and misleading. Unfortunately, this misunderstanding continues to propagate prior misinformation disseminated about our work published in the 2015 NEJM paper. As clearly stated in our 2015 manuscript, we discovered and measured the levels of formaldehyde hemiacetals, not gaseous carbonyl formaldehyde. The 2017 study by Farsalinos et al. only measured levels of carbonyl formaldehyde. The ongoing omission of this critical detail, as reflected in the posts below as well as in the recent claimed “replication” of our work, has led the authors (yet again) to the conclusion that high levels of formaldehyde are detectable by e-cigarette users only under “dry puff” conditions. As we have described in a very recent manuscript (https://www.nature.com/articles/s41598-017-11499-0), 35-45 % of the formaldehyde hemiacetals are not converted to the detectable form of formaldehyde using DNPH impinger or related sorbent tube methods. Accounting for an approximately 40% loss of detectable formaldehyde at a power level of 4V, which the authors described as “realistic use conditions,” affords levels of > ca. 1400 ug of formaldehyde consumed. This is essentially the same level of formaldehyde from smoking combustible cigarettes, again according to data in the manuscript by Farsalinos and co-workers. This means that the cancer risk from formaldehyde in the e-cigarette that was investigated in the so-called replication study, even at this modest power level, is the same as that for combustible cigarettes. In other words, e-cigarettes do not emit very high formaldehyde levels only in conditions that are averse to users. In addition, discussing ‘dry puffs’ with the study subjects and increasing the power levels in a non-random way do not seem best practice for generating unbiased human subject study results. Our further studies of the chemistry and health effects of e-cigarettes are ongoing, and will be reported in due course, in peer-reviewed journals.

On 2017 Sep 02, Clive Bates commented:

This study has now been replicated and its obvious flaws confirmed.

Farsalinos KE, Voudris V, Spyrou A, Poulas K. E-cigarettes emit very high formaldehyde levels only in conditions that are aversive to users: A replication study under verified realistic use conditions. Food Chem Toxicol. 2017; . [link]

Conclusion

The high levels of formaldehyde emissions that were reported in a previous study were caused by unrealistic use conditions that create the unpleasant taste of dry puffs to e-cigarette users and are thus avoided.

The high levels of formaldehyde reported by the authors occur only in "dry puff" conditions that would be too acrid and aversive for human users to withstand. The authors completely missed this human control mechanism yet went on to calculate cancer risks based on conditions that humans would never experience. The resulting media storm was deeply misleading and the result of the authors inappropriate reporting of cancer risks based on unrealistic exposures.

This research letter should have been withdrawn or corrected by the NEJM once the flaws were pointed out. Given the controversy and impact on public health arising from people being given misleading information about health risks, the journal should have proactively commissioned a replication study or at least published this one.

A correction by the authors could at least admit that cancer is a human condition and that the cancer risk calculations have no validity given that humans are not exposed toxins generated under dry puff conditions.

At the time of writing the paper has 138 academic citations and 386 media articles. How many people will have been misled into making decisions harmful to their health by this paper?

On 2016 Apr 20, Clive Bates commented:

The fundamental weakness in this study is a failure to control the vapourising temperature to the levels experienced by real human subjects, rather operating temperatures that create a sensory experience that can only be tolerated by laboratory machines. The artifically high operating temperatures account for the high levels of formaldehyde and similar detected in the study.

At high temperatures, e-cigarette vapour undergoes thermal decomposition and aldehydes, such as formaldehyde, will form. For human subjects, there is a control feedback that avoids this exposure - namely that the aerosol takes on a horrible harsh taste and the e-cigarette user stops vaping. This phenomenon is widely understood among users and is known as 'dry puff' or 'dry hit' and it is always avoided. No such control feedback exists for laboratory machines.

This is one of several studies that ignores this important effect - something that could have been easily remedied by involving human users to verify the puffing protocol and device configuration and voltage settings create a realistic human vaping experience.

While it is not incorrect, merely pointless, to measure and report the decomposition of vapour in these unrealistic conditions, it is not legitimate to go on to make and publicise calculations of cancer risk, which can only arise from human exposures.

A critique of this short but damaging paper was published in the journal Addiction with an exchange of letters (Disclosure - I am a co-author): Bates CD, Farsalinos KE. Research letter on e-cigarette cancer risk was so misleading it should be retracted. Addiction 2015;110:1686–7. doi:10.1111/add.13018 Link to critique in Addiction

A detailed critique and case for retraction under the COPE guidelines was provided and published as supplementary material in Addiction. Link to Complaint under the Code of Conduct of the Committee on Publication Ethics

Forty experts wrote to the New England Journal of Medicine to express concern about the paper. Their letter was also published as supplementary material in Addiction. Link to letter

The New England Journal of Medicine has not provided a substantive response to this complaint or offered a route for escalation at the journal, as it should do under COPE. The full engagement with NEJM is described here with a annotated document file here - Google Doc

On 2015 Sep 14, Ivan Oransky commented:

Critics have called for this paper to be retracted, and there has been extensive correspondence about it in Addiction: http://retractionwatch.com/2015/09/11/researchers-call-for-retraction-of-nejm-paper-showing-dangers-of-e-cigarettes/

On 2015 Sep 14, Ivan Oransky commented:

Critics have called for this paper to be retracted, and there has been extensive correspondence about it in Addiction: http://retractionwatch.com/2015/09/11/researchers-call-for-retraction-of-nejm-paper-showing-dangers-of-e-cigarettes/

On 2016 Apr 20, Clive Bates commented:

The fundamental weakness in this study is a failure to control the vapourising temperature to the levels experienced by real human subjects, rather operating temperatures that create a sensory experience that can only be tolerated by laboratory machines. The artifically high operating temperatures account for the high levels of formaldehyde and similar detected in the study.

At high temperatures, e-cigarette vapour undergoes thermal decomposition and aldehydes, such as formaldehyde, will form. For human subjects, there is a control feedback that avoids this exposure - namely that the aerosol takes on a horrible harsh taste and the e-cigarette user stops vaping. This phenomenon is widely understood among users and is known as 'dry puff' or 'dry hit' and it is always avoided. No such control feedback exists for laboratory machines.

This is one of several studies that ignores this important effect - something that could have been easily remedied by involving human users to verify the puffing protocol and device configuration and voltage settings create a realistic human vaping experience.

While it is not incorrect, merely pointless, to measure and report the decomposition of vapour in these unrealistic conditions, it is not legitimate to go on to make and publicise calculations of cancer risk, which can only arise from human exposures.

A critique of this short but damaging paper was published in the journal Addiction with an exchange of letters (Disclosure - I am a co-author): Bates CD, Farsalinos KE. Research letter on e-cigarette cancer risk was so misleading it should be retracted. Addiction 2015;110:1686–7. doi:10.1111/add.13018 Link to critique in Addiction

A detailed critique and case for retraction under the COPE guidelines was provided and published as supplementary material in Addiction. Link to Complaint under the Code of Conduct of the Committee on Publication Ethics

Forty experts wrote to the New England Journal of Medicine to express concern about the paper. Their letter was also published as supplementary material in Addiction. Link to letter

The New England Journal of Medicine has not provided a substantive response to this complaint or offered a route for escalation at the journal, as it should do under COPE. The full engagement with NEJM is described here with a annotated document file here - Google Doc

On 2017 Sep 02, Clive Bates commented:

This study has now been replicated and its obvious flaws confirmed.

Farsalinos KE, Voudris V, Spyrou A, Poulas K. E-cigarettes emit very high formaldehyde levels only in conditions that are aversive to users: A replication study under verified realistic use conditions. Food Chem Toxicol. 2017; . [link]

Conclusion

The high levels of formaldehyde emissions that were reported in a previous study were caused by unrealistic use conditions that create the unpleasant taste of dry puffs to e-cigarette users and are thus avoided.

The high levels of formaldehyde reported by the authors occur only in "dry puff" conditions that would be too acrid and aversive for human users to withstand. The authors completely missed this human control mechanism yet went on to calculate cancer risks based on conditions that humans would never experience. The resulting media storm was deeply misleading and the result of the authors inappropriate reporting of cancer risks based on unrealistic exposures.

This research letter should have been withdrawn or corrected by the NEJM once the flaws were pointed out. Given the controversy and impact on public health arising from people being given misleading information about health risks, the journal should have proactively commissioned a replication study or at least published this one.

A correction by the authors could at least admit that cancer is a human condition and that the cancer risk calculations have no validity given that humans are not exposed toxins generated under dry puff conditions.

At the time of writing the paper has 138 academic citations and 386 media articles. How many people will have been misled into making decisions harmful to their health by this paper?

On 2017 Sep 14, David H Peyton commented:

We appreciate the feedback on our manuscript, and respect the concerns of the posters here as well as those of the authors of the 2017 Food Chem Toxicol. paper by Farsalinos et al., cited by Clive Bates below. We share in the common goal of maximizing harm reduction. However, the claim that the aforementioned manuscript embodies a “replication” of our study in NEJM is false and misleading. Unfortunately, this misunderstanding continues to propagate prior misinformation disseminated about our work published in the 2015 NEJM paper. As clearly stated in our 2015 manuscript, we discovered and measured the levels of formaldehyde hemiacetals, not gaseous carbonyl formaldehyde. The 2017 study by Farsalinos et al. only measured levels of carbonyl formaldehyde. The ongoing omission of this critical detail, as reflected in the posts below as well as in the recent claimed “replication” of our work, has led the authors (yet again) to the conclusion that high levels of formaldehyde are detectable by e-cigarette users only under “dry puff” conditions. As we have described in a very recent manuscript (https://www.nature.com/articles/s41598-017-11499-0), 35-45 % of the formaldehyde hemiacetals are not converted to the detectable form of formaldehyde using DNPH impinger or related sorbent tube methods. Accounting for an approximately 40% loss of detectable formaldehyde at a power level of 4V, which the authors described as “realistic use conditions,” affords levels of > ca. 1400 ug of formaldehyde consumed. This is essentially the same level of formaldehyde from smoking combustible cigarettes, again according to data in the manuscript by Farsalinos and co-workers. This means that the cancer risk from formaldehyde in the e-cigarette that was investigated in the so-called replication study, even at this modest power level, is the same as that for combustible cigarettes. In other words, e-cigarettes do not emit very high formaldehyde levels only in conditions that are averse to users. In addition, discussing ‘dry puffs’ with the study subjects and increasing the power levels in a non-random way do not seem best practice for generating unbiased human subject study results. Our further studies of the chemistry and health effects of e-cigarettes are ongoing, and will be reported in due course, in peer-reviewed journals.

On 2017 Sep 18, Konstantinos Farsalinos commented:

Although Jensen et al. mentioned in the 2015 NEJM research letter that the health risks of formaldehyde hemiacetal inhalation are unknown ("How formaldehyde-releasing agents behave in the respiratory tract is unknown..."), they made a calculation that the formaldehyde-attributable cancer risk from e-cigarette use is 5 to 15 times higher than from long-term smoking. These two statements are clearly contradictory, and the calculation of any cancer risk from formaldehyde hemiacetal emissions is invalid since no such risk has been established for these compounds. It is simply based on an unsubstantiated assumption that "inhaling formaldehyde-releasing agents carries the same risk per unit of formaldehyde as the risk associated with inhaling gaseous formaldehyde...". Additionally, the latter statement was further misinterpreted by the media that reported the cancer risk of e-cigarettes (total risk, not formaldehyde-attributable risk) being 5 to 15 times higher than smoking. This is extremely important because, according to Fowles and Dybing (http://tobaccocontrol.bmj.com/content/12/4/424.long), the formaldehyde-attributable cancer risk is less than 1% of the total cancer risk of smoking. Prof Peyton, one of the authors of the NEJM research letter, now claims that formaldehyde hemiacetals are different from gaseous formaldehyde. But then, why did he and his co-authors use the hemiacetal measurements to present the cancer risk of e-cigarettes relative to tobacco cigarettes, considering that literature data for gaseous formaldehyde emissions from tobacco cigarettes were used in the comparison?

In the recently published paper (https://www.nature.com/articles/s41598-017-11499-0), Salamanca et al. are comparing the results of an experimental method (NMR) with the results of an established and validated analytical method used to identify and measure aldehyde emissions. In our study, using a validated and widely accepted method, we found 89% higher levels of formaldehyde at 5 V than the level of formaldehyde hemiacetal reported by Jensen et al. in the NEJM research letter. Therefore, we did not underestimate formaldehyde levels. We did not reject the findings by Jensen et al. that, high formaldehyde levels can be produced when blindly testing e-cigarettes at inappropriately high power settings relevant to the atomizer used (in fact we found more, but in general our findings support their observations). However, this cannot be used as an indication that e-cigarette users are exposed to such extreme formaldehyde levels in their daily routine.

I would also like to remind that we used a product which is outdated and inefficiently designed, a design that has been abandoned since 2012. These products are not even available in the market in the European Union anymore. Recent atomizers release formaldehyde at levels orders of magnitude lower than the one we tested. Even if we assume, despite the lack of evidence, that formaldehyde hemiacetals carry the same health risk as formaldehyde and that DNP derivatization underestimates the total formaldehyde exposure, still e-cigarettes developed over that last 3 years (at least) emit by far lower formaldehyde than tobacco cigarettes. Finally, the comparison between tobacco and electronic cigarettes in formaldehyde emissions is based on measurements performed using the same analytical method (DNPH derivatization) for both products. The assumption about underestimation of formaldehyde by DNPH derivatization applies equally to tobacco cigarettes and electronic cigarettes (tobacco cigarettes contain a lot of PG and VG); thus the relative difference remains the same.

Recent studies have found that e-cigarettes release formaldehyde at levels corresponding to > 1900 cigarettes consumed per day (e.g. https://www.ncbi.nlm.nih.gov/pubmed/27461870). All these studies, which have been accompanied by press statements and wide media coverage, need to be replicated under verified realistic conditions. Science is defined by replication, and we hope that journal editors will accept the challenge of publishing studies that reject findings previously reported in the same journal. In any case, all our replication studies will be published in peer-reviewed journals.

In conclusion, our purpose was to replicate an experiment using a specific e-cigarette device, atomizer and liquid under verified realistic conditions in order to verify or reject the conclusion that e-cigarettes carry 5 to 15 times the cancer risk of tobacco cigarettes. Our study clearly identified that this statement is false, and this was evident even when we used a very outdated and inefficient e-cigarette product. Therefore, our study was a valid replication experiment that presented the clinical relevance of previous findings.