On 2016 Jul 01, Andrea Messori commented:

Structured methods of analysis and reporting: a quick literature analysis

Sabrina Trippoli and Andrea Messori

HTA Unit, ESTAR, Regional Health Service, 50100 Firenze (Italy)

In the evaluation of therapeutic interventions, structured methods of analysis and reporting are increasingly being proposed (1-6). Among these, the GRADE method (4-6), firstly described in 2004, is the most widely recognised at international level. For example, Alonso-Coello and colleagues (7) have pointed out that the “more than 100 organisations globally, including the World Health Organization, the Cochrane Collaboration, and the National Institute for Health and Care Excellence now use or have adopted the principles of the GRADE system”. In the present comment, we report the results of a preliminary literature analysis that we conducted to explore the contents of the published articles dealing with the GRADE method. Our analysis was restricted to journals indexed by PubMed (www.pubmed.gov).

a) Search engine: PubMed

b) Search term: “GRADE method”

c) Date of the search: 1 July 2016

d) Extracted citations: N=71

e) No. of citations not pertinent with the search term (false positive): N=17

f) No. of citations pertinent with the search term: N=54

g) No. of citations the contents of which were restricted to a description of the GRADE method and/or the intention to use this method for future activities: N=12

h) No. of citations the contents of which were represented by an experience of application of the method along with the results of this application: N=42. Of these 42 papers, 32 used the GRADE method with the only purpose of assessing the quality of evidence of the clinical material included in a systematic review or a meta-analysis; the remaining 10 of these 42 papers applied the GRADE method to the development of clinical guidelines or recommendations.

References

1. Schnipper LE, Davidson NE, Wollins DS, et al: American Society of Clinical Oncology statement: A conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:2563-2577.

2. Schnipper LE, Davidson NE, Wollins DS, et al: Updating the ASCO value framework: Revisions and reflections in response to comments received. J Clin Oncol 2016, doi: 10.1200/JCO.2016.68.251

3. Cherny NI, Sullivan R, Dafni U, et al: A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: The European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 2015;26:1547-1573.

4. Atkins D, Best D, Briss PA, et al. GRADE Working Group. Grading quality of evidence and strength of recommendations.BMJ 2004;328:1490.

5. Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6.

6. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol2011;64:383-94.

7. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD; GRADE Working Group. GRADE Evidence to Decision (EtD) frameworks: asystematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016 Jun 28;353:i2016.

Address correspondence to: Dr.Andrea Messori, HTA Unit, ESTAR, Regional Health Service, 50100 Firenze (Italy)

On 2017 Dec 26, Kevin Kavanagh commented:

We have expressed concerns in a previous letter and PubMed Common’s posting regarding the article by Kelly, et al,(1) where the efficacy of the evaluated product may have been overstated.(2, 3) In the authors reply, data was presented which presents less than a 30% reduction(4) as opposed to a 42% which is stated in the article and advertised by the company.(1,5) To our knowledge the peer-review record has not been corrected. In addition, we have concerns regarding at least the appearance of an undeclared conflict-of-interest between one of the article’s authors, Connie Steed, and the company in question, DebMed.(6)

It has come to the authors’ attention that the editor in charge of adjudicating the above concerns may also have a conflict-of-interest with DebMed and with one of the authors of the manuscript in question. Significant concerns regarding the conflicts of interest of the Editor Elaine Larson have arisen because of the following associations:

• Co-Author with Connie Steed (one of the authors in the manuscript in question) and Paul Alpert (Vice President of Patient Safety Strategy for DebMed) in an article published in Feb 2011.(7). Conflicts-of-Interest stated the following “Elaine Larson has received research funding from Deb Worldwide Healthcare, Inc.”

• Co-Author with Paul Alpert (Vice-President of Patient Safety Strategy for DebMed) in an article published in Jan 2013.(8)

• Co-Author with Paul Alpert (Vice-President of Patient Safety Strategy for DebMed) in an article published in Feb 2014.(9)

• Connie Steed, RN is listed as the 2016 Secretary for the Association for Professionals in Infection Control and Epidemiology, Inc., which has as its official publication the American Journal of Infection Control(10) and provides this Journal as a benefit of their membership.(11)

We feel that because of the above, the appearance of a conflict of interest exists which may have clouded the decision making and inhibited the correction of the potential research integrity problems in the article in question.

References

(1) Kelly, J.W., Blackhurst, D., McAtee, W., and Steed, C. Electronic hand hygiene monitoring as a tool for reducing health care-associated methicillin-resistant Staphylococcus aureus infection. Am J Infect Control. 2016; 44: 956–95 Kelly JW, 2016

(2) Kavanagh KT, Saman DM. Comment Regarding: Electronic hand hygiene monitoring as a tool for reducing health care–associated methicillin-resistant Staphylococcus aureus infection. American Journal of infection Control. December 01 2016 http://www.ajicjournal.org/article/S0196-6553(16)30904-X/fulltext

(3) Kavanagh KT, Saman DM. Comment on PMID: 27908437: Response to Letter Regarding Manuscript “Electronic Hand Hygiene Monitoring as a Tool for Reducing Nosocomial Methicillin-resistant Staphylococcus aureus Infection” In: PubMed Commons [Internet]. Bethesda (MD): National Library of Medicine; 2016 Dec. 16. Available from: https://www.ncbi.nlm.nih.gov/pubmed/27908437#cm27908437_34333

(4) Kelly JW, Blackhurst D, McAtee W, Steed C. Response to Letter Regarding Manuscript "Electronic Hand Hygiene Monitoring as a Tool for Reducing Nosocomial Methicillin-resistant Staphylococcus aureus Infection". Am J Infect Control. 2016 Dec 1;44(12):1763. doi>: 10.1016/j.ajic.2016.08.009. http://www.ajicjournal.org/article/S0196-6553(16)30812-4/fulltext Kelly JW, 2016

(5) DebMed: Discover how one facility was able to reduce MRSA HAIs by up to 42%. Last accessed on Dec. 21, 2017 from http://info.debmed.com/mrsa-study-flyer

(6) Hospital Reduces MRSA Rates by 42% with electronic hand hygiene measurement. Infection Control Today. July 8, 2016. http://www.infectioncontroltoday.com/news/2016/07/hospital-reduces-mrsa-rates-by-42-with-electronic-hand-hygiene-measurement.aspx

(7) Connie Steed, MSN, RN, CIC J. William Kelly, MD Dawn Blackhurst, DrPH Sue Boeker, BSN, RN, CIC Thomas Diller, MD, MMM Paul Alper, BA Elaine Larson, RN, PhD, FAAN, CIC Hospital hand hygiene opportunities: Where and when (HOW2)? The HOW2 Benchmark Study. American Journal of Infection Control. Feb 2011 39(1):19-26. Steed C, 2011

(8) Buet A, Cohen B, Marine M, Scully F, Alper P, Simpser E, Saiman L, Larson E. Hand hygiene opportunities in pediatric extended care facilities. J Pediatr Nurs. 2013 Jan;28(1):72-6. doi: 10.1016/j.pedn.2012.04.010. Epub 2012 Jun 1. Buet A, 2013

(9) Conway LJ, Riley L, Saiman L, Cohen B, Alper P, Larson EL. Implementation and impact of an automated group monitoring and feedback system to promote hand hygiene among health care personnel. Jt Comm J Qual Patient Saf. 2014 Sep;40(9):408-17. Conway LJ, 2014

(10) American Journal of Infection Control Home Page. Accessed on Dec. 26, 2017 from https://www.journals.elsevier.com/ajic-american-journal-of-infection-control

(11) Association for Professionals in Infection Control and Epidemiology, Inc. Web Posting. Accessed on Dec. 26, 2017 from http://www.ajicjournal.org/article/S0196-6553(15)01270-5/pdf

Comment also posted on PubPeer

On 2017 Nov 30, Natalie Clairoux commented:

Free version of this article available after March 29, 2018 at https://papyrus.bib.umontreal.ca/xmlui/handle/1866/19235

On 2016 Jun 25, Christopher Southan commented:

A nicely detailed paper but it seems odd the structures were not deposited in PDB since the coordinates (460 pages!) are already published in WO2015200795. CC-885 is now entered into Guide to PHARMACOLOGY as ligand ID 9224 (live link pending in July release)

On 2016 Jun 30, Atanas G. Atanasov commented:

For broader view on the significance of natural products in drug discovery and development, you might also check:

http://dx.doi.org/10.1016/j.biotechadv.2015.08.001

(“Discovery and resupply of pharmacologically active plant-derived natural products: A review”)

On 2016 Jun 27, L. Miguel Martins commented:

We have a video abstract for this paper here: https://youtu.be/13oA5eiH-8s

On 2016 Oct 24, Sean Ekins commented:

THis work gets a mention alongside other machine learning efforts http://www.slideshare.net/ekinssean/using-machine-learning-models-based-on-phenotypic-data-to-discover-new

On 2016 Jul 10, Marios Kyriazis commented:

There is a fundamental issue which many people either forget or choose to ignore. The effort to reduce, reverse or postpone aging should not reflect our innate wish to 'live longer'.Instead it reflects a medical aim to reduce, eliminate or avoid age-related, chronic degeneration, which leads to dysfunction. It is not a matter of simply accepting the finitude of life, because this finitude is necessarily associated with chronic dysfunction and disease which would not be present if we find ways to avoid aging.

On 2016 Aug 26, Sharanbasappa Durg commented:

The treatment of low back pain (LBP) is challenging and guidelines recommend medications with proven benefits. Further, patients’ preference should be considered in the treatment of pain. Systematic reviews (mainly of randomized controlled trials [RCTs]), with or without meta-analysis, are generally considered to provide the key evidence in the practice of evidence-based medicine.

The systematic review and meta-analysis by Abdel Shaheed C, 2017 pooled all the available clinical evidence (of RCTs) on the use of muscle relaxants (Thiocolchicoside, Carisoprodol, Tizanidine, Eperisone, Pridinol, Flupirtine, Cyclobenzaprine) for LBP. Most of the included trials evaluated the efficacy and safety of muscle relaxants in acute LBP participants. The controls were mainly either placebo or nonsteroidal anti-inflammatory drugs (NSAIDs). The study authors could have tried some meta-analysis on important physiological outcomes. This systematic review concluded that muscle relaxants provide clinically significant pain relief for acute LBP.

Similar outcomes were observed in a systematic literature review on Eperisone for LBP by Bavage S, 2016, 2016. In this systematic review, the authors found that intervention with Eperisone may be effective in acute LBP patients with less adverse effects. Further, Eperisone also improved paraspinal blood flow in chronic LBP patients.

Both the systematic reviews, however, did not find any considerably support for the use of muscle relaxants in chronic LBP patients. Though, few RCTs evaluated the efficacy and safety of muscle relaxants in chronic LBP. Future research should emphasize more on finding the clinical evidence whether muscle relaxants have beneficial effects in patients with chronic LBP.

On 2016 Aug 26, Sharanbasappa Durg commented:

None

On 2016 Aug 25, Gerard Ridgway commented:

There is some very interesting discussion of this paper on AlzForum. In particular, Pieter Jelle Visser reports a strong correlation (0.8) between CSF Aβ1-42 and amyloid PET, and David Holtzman notes that CSF Aβ1-42 drops at or before detectable PiB positivity, and that PiB is in turn more sensitive than florbetapir. Another recent paper using ADNI data (Palmqvist S, 2016) also concludes that CSF Aβ1-42 "becomes abnormal in the earliest stages of Alzheimer's disease, before amyloid positron emission tomography and before neurodegeneration".

These results seem strongly discordant with Figure 3 in this paper, which shows florbetapir Aβ deposition occurring second (after vascular abnormalities) and becoming fully abnormal in established LOAD, but CSF Aβ1-42 abnormalities occurring last (out of 9 biomarkers), and reaching an abnormality level of only about 1/3. If this difference between amyloid PET and CSF Aβ1-42 results is considered implausible, then that could suggest a modelling problem that could also call into question the key finding of early vascular abnormalities.

On 2017 Nov 27, Eyal Shahar commented:

"The consistency thingamajig is a bundle of biases rolled into one. First, it is the assumption that effect‐modification is absent (which is strange by itself because effect modification plays a bizarre role in determinism.) Second, the claim that exposure needs to be “defined” either leads to thought bias or illustrates misunderstanding of information bias."

The consistency rule http://www.u.arizona.edu/~shahar/commentaries.html

On 2017 Jun 09, GERALD SMITH commented:

In Ma et al. (1), we isolated Schizosaccharomyces pombe ctp1 point mutations, which we interpreted to separate genetically two known meiotic activities that depend on Ctp1 for meiotic DNA break repair. A later paper by Jensen and Russell (2) claimed these activities are not genetically separable. We believe the disparity in interpretation stems largely from the use of meiotic cells in our studies but mitotic cells by Jensen and Russell. We maintain that in meiotic cells the activities are genetically separable.

During S. pombe meiosis, Rec12 (Spo11 homolog) makes DNA double-strand breaks (DSBs) and remains covalently linked to each 5’ end. The Mre11-Rad50-Nbs1 (MRN)-Ctp1 complex endonucleolytically removes Rec12 covalently linked to a short oligonucleotide (Rec12-oligo), an activity called “clipping.” The recessed 5’ end is then further digested, an activity called “resection,” to form a long 3’ single-stranded DNA tail that forms with intact DNA a joint DNA molecule to continue DSB repair. We concluded that most of the dozen ctp1 point mutants we studied retained nearly wild-type levels of resection but little clipping activity during meiosis. These conclusions were based on concordant genetic and physical analyses. Although we isolated these mutants based on a mitotic screen, we drew all of our conclusions from meiotic data.

In our physical assays for clipping, we found in wild-type cell extracts abundant Rec12-oligos, which were absent or barely detectable in ctp1 mutant extracts, either point mutant or complete deletion (ctp1Δ). Failure to clip off Rec12 leaves irreparable DSBs and consequently inviable spores; the ctp1 point mutants had 10- to 30,000-fold, and ctp1Δ 100,000-fold, reductions of viable spore yields compared to wild type. These results indicate that the ctp1 mutants have strongly reduced clipping activity.

To assay resection, we induced in meiotic cells a DSB at a well-defined site, using either the I-SceI or the I-PpoI homing endonuclease. Physical assays using Southern blots showed that resection of the DSB end proceeded as rapidly and extensively in the ctp1 point mutants as in wild type but slowly and to a much lesser extent in the ctp1Δ mutant. The homing endonucleases form DSBs without a covalently bound protein; I-SceI-dependent recombination thus requires resection but not clipping. In the ctp1 point mutants, the frequency of this recombination was equal to, or even twice as high as, that in ctp1+ cells, which was ~5 times higher than that in the ctp1Δ mutant. Thus, both physical and genetic assays indicate that resection is robust in the ctp1 point mutants.

In interpreting these results, it is important to consider the role of Exonuclease I (ExoI), which could potentially resect DSB ends and produce recombinants. The MRN complex blocks ExoI access to DSBs in meiotic cells, but the Ku complex blocks ExoI in mitotic cells. Thus, ExoI plays no apparent role in DSB repair in meiotic cells with an intact MRN complex; instead, MRN promotes access of Ctp1 to DSB ends. For example, in meiotic cells exoIΔ has no significant effect on I-SceI-dependent recombination, but ctp1Δ reduces it by a factor of ~5 (3). Conversely, in rad50Δ mutants ctp1Δ has no significant effect, but exoIΔ reduces recombination by a factor of ~4. As expected, the ctp1Δ exoIΔ double mutant is like ctp1Δ in rad50+ cells but like exoIΔ in rad50Δ mutants.

Jensen and Russell (2) assessed the mitotic activity of two of our ctp1 point mutants, along with ctp1+ and ctp1Δ, by analyzing the number and sizes of colonies formed on agar plates containing DNA damaging agents and spotted with 5-fold serial dilutions of cell cultures. Their results, like ours, showed that ctp1-6 and ctp1-25 are more DNA damage-resistant than the ctp1Δ mutant, indicating that the point mutants retain some activity. Removal of the Ku complex, by pku80Δ, suppressed these sensitivities but only if ExoI was present. Removal of ExoI enhanced the sensitivity to some agents (e.g., methyl methanesulfonate) but not, or only slightly, to others (e.g., ionizing radiation). In each of these various combinations, the point mutants were more resistant than the ctp1Δ mutant, confirming our results that the point mutants retain some activity. In nine out of ten cases, the ctp1-6 exoIΔ and ctp1-25 exoIΔ mutants were more resistant than ctp1Δ exoIΔ, which was completely sensitive to the agents tested. This result shows that the ctp1 point mutants retain an activity that can be supplied by ExoI, which we take to be resection.

Jensen and Russell proposed that the ctp1 point mutations reduce, but do not abolish, a single activity. They further proposed that this residual activity is sufficient to repair a single DSB per cell (as in the I-SceI and I-PpoI experiments we reported) but not multiple DSBs per cell [as in the experiments with wild-type Rec12, which makes about 60 DSBs per meiotic cell (4)]. This proposal is hard to reconcile with our physical assays of clipping and resection noted above. It is not clear to us how a single Ctp1 activity could be altered, in either KM or kcat, to resect one DSB in a cell with wild-type kinetics but not clip Rec12 off 60 DSBs in a cell if resection and clipping result from the same activity, since both processes take about the same amount of time. Rather, we think the mutant proteins have greater reductions of clipping activity than of resection activity. Reduction of the number of Ctp1 molecules per cell, from 60 or more to about 1, could account for our results, but we consider this an unlikely explanation for the many mutants we studied. Instead, we think our mutants have retained nearly wild-type levels of resection but have strongly reduced levels of clipping, in meiotic cells. Whether the active sites for these two activities are in Ctp1, the MRN complex, or both is not addressed by our experiments. Without tests of a very large number of Ctp1 mutants under many conditions, we think the conclusion that the two activities are not genetically separable is unwarranted.

Lijuan Ma, Neta Milman, Mridula Nambiar, and Gerald R. Smith

References:

1. Ma, L., Milman, N., Nambiar, M. and Smith, G.R. (2015) Two separable functions of Ctp1 in the early steps of meiotic DNA double-strand break repair. Nucleic Acids Res., 43, 7349-7359.

2. Jensen, K.L. and Russell, P. (2016) Ctp1-dependent clipping and resection of DNA double-strand breaks by Mre11 endonuclease complex are not genetically separable. Nucleic Acids Res., 44, 8241-8249.

3. Farah, J.A., Cromie, G.A. and Smith, G.R. (2009) Ctp1 and Exonuclease 1, alternative nucleases regulated by the MRN complex, are required for efficient meiotic DNA repair and recombination. Proc. Natl. Acad. Sci. USA, 106, 9356-9361.

4. Fowler, K.R., Sasaki, M., Milman, N., Keeney, S. and Smith, G.R. (2014) Evolutionarily diverse determinants of meiotic DNA break and recombination landscapes across the genome. Genome Res., 24, 1650-1664.

On 2017 Sep 20, Jim Woodgett commented:

Since GSK-3alpha is equally inhibited by lithium, the power of this study may have been increased if phosphorylation of Serine 21 of the alpha isoform had been measured too.

On 2016 Jul 21, David Keller commented:

Aluminum exposure raised dementia risk by 71% in meta-analysis. 44% PPI dementia risk needs correction for aluminum-containing antacids!

Dietary aluminum ingestion is theorized to be neurotoxic and play a causative role in the onset and progression of dementia. [1-3] A recent meta-analysis showed that individuals chronically exposed to aluminum were 71% more likely to develop Alzheimer disease (odds ratio, 1.71; 95% CI, 1.35-2.18).[4] Many strong antacids contain aluminum hydroxide and are often taken for years by patients with peptic ulcer disease or gastroesophageal reflux before they are prescribed proton pump inhibitors, and concurrent with their use. Gomm and colleagues [5] did not correct for the use of aluminum-containing antacids when calculating the association of dementia with proton pump inhibitor use. How much of their observed association of proton pump inhibitor use with dementia is actually due to long-term ingestion of aluminum antacids, either currently or in the past?

References

1: Bhattacharjee S, Zhao Y, Hill JM, Percy ME, Lukiw WJ. Aluminum and its potential contribution to Alzheimer's disease (AD). Front Aging Neurosci. 2014 Apr 8;6:62. doi: 10.3389/fnagi.2014.00062. eCollection 2014. PubMed PMID:24782759; PubMed Central PMCID: PMC3986683.

2: Rodella LF, Ricci F, Borsani E, Stacchiotti A, Foglio E, Favero G, Rezzani R, Mariani C, Bianchi R. Aluminium exposure induces Alzheimer's disease-like histopathological alterations in mouse brain. Histol Histopathol. 2008 Apr;23(4):433-9. PubMed PMID: 18228200.

3: Exley C. What is the risk of aluminium as a neurotoxin? Expert Rev Neurother. 2014 Jun;14(6):589-91. doi: 10.1586/14737175.2014.915745. Epub 2014 Apr 30. PubMed PMID: 24779346.

4: Wang Z, Wei X, Yang J, Suo J, Chen J, Liu X, Zhao X. Chronic exposure to aluminum and risk of Alzheimer's disease: A meta-analysis. Neurosci Lett. 2016 Jan 1;610:200-6. doi:10.1016/j.neulet.2015.11.014. Epub 2015 Nov 27. PubMed PMID: 26592479.

5: Gomm W, von Holt K, Thomé F, et al. Association of proton pump inhibitors with risk of dementia: a pharmacoepidemiological claims data analysis [published online February 15, 2016]. JAMA Neurol. doi:10.1001/jamaneurol.2015.4791.

On 2016 Aug 03, Murat Büyükşekerci commented:

Dear Sir In Page 26, under title "5.1 Pain Chronification", it is written as 'For example, neurokinin has been shown to both decrease inflammatory pain and joint cartilage destruction in rats [176, 177]. According to referenced atricles it should be like this '..NEUROKİNİN RECEPTOR ANTAGONİSTS.." not neurokinin. Thanks.

On 2016 Aug 02, Murat Büyükşekerci commented:

In page 13 under heading 3.8 Glutamate it is written as "They showed that alpha-2 antagonists inhibit the auto-phosphorylation of CaMKII, proposing that reduced CaMKII activity may lead to dephosphorylation of NMDA and AMPA glutamate receptors and a corresponding decrease in nociceptive transmission [77]." However, as we read the referenced article* clonidin ,an alpha-2 agonist, did inhibit the auto-phosphorylation of CaMKII but not any alpha-2 antagonists. I would like to point this misinterpret. Thank you for considering.

*Wang XT, Lian X, Xu YM, et al. Alpha2 noradrenergic receptor suppressed CaMKII signaling in spinal dorsal horn of mice with inflammatory pain. Eur J Pharmacol 2014;724:16-23.

On 2017 Dec 10, Ricardo Belda commented:

Any subclinical leaks revealed by the tomography? Congrats

On 2018 Jan 12, Stefan Tino Kulnik commented:

Further to my comment from 13 October 2017:

In June 2017, we approached Journal of Physiotherapy and submitted a commentary, in which we pointed out the error in this meta-analysis and that the authors’ conclusion with respect to the impact of respiratory muscle training on respiratory complications is therefore unfounded. Unfortunately our commentary was rejected, and so we were denied the opportunity of entering a scientific exchange with the authors within the pages of the journal.

The journal did acknowledge the data extraction error we pointed out and promised a correction, but a correction has not been published to date.

It may be regarded as rather unfortunate that this systematic review and meta-analysis (a study design that many colleagues in clinical practice will view as highest level evidence) presents a strong clinical message in favour of implementing respiratory muscle training for the prevention of respiratory complications, based on an erroneous meta-analysis.

Burden of treatment and opportunity cost to stroke survivors should not be underestimated, and it is important to focus clinical resources on the most meaningful rehabilitation activities based on best evidence.

For members of the research community and colleagues in clinical practice who may be interested, I have uploaded the content of our rejected commentary on my ResearchGate page https://tinyurl.com/yb3wxmkf. In this we also present a re-calculated meta-analysis using Peto odds ratio, which is a more appropriate and statistically more powerful model of meta-analysis when events are rare, to demonstrate that even with this statistically more powerful method the meta-analysis still fails to reach statistical significance of the overall effect.

On 2017 Oct 13, Stefan Tino Kulnik commented:

I write to highlight an error in this systematic review and meta-analysis of respiratory muscle training after stroke. The authors present a meta-analysis, in which data from two randomised controlled trials of respiratory muscle training in stroke were pooled (Figure 6). Respiratory muscle training is reported to result in a statistically significant risk reduction for the outcome respiratory complications (overall risk ratio 0.38, 95%CI 0.15 to 0.96).

It is this journal’s policy to publish simplified forest plots in the main article, and to provide detailed forest plots in an online supplement. It is therefore not immediately evident that in this meta-analysis there has been a data extraction error from one of the included studies.

The detailed forest plot for this analysis (Figure 7 in the online supplement) shows that for meta-analysis two intervention groups in the study by Kulnik ST, 2015 were combined to a total of n=53. The number of respiratory complications in this combined group is given as n=5, but in fact it was n=9. The correct overall risk ratio in meta-analysis (Mantel-Haenszel random effects model in RevMan, Version 5.3, 2014) is therefore 0.49 (95%CI 0.09 to 2.65). While this re-calculated point estimate still favours the intervention, it is no longer statistically significant.

This demonstrates how in studies such as these, where sample sizes are small and events are rare, small inaccuracies can have a considerable influence on statistical results.

Further studies are required to demonstrate a statistically significant effect of respiratory muscle training on respiratory complications after stroke. There is a good theoretical rationale why respiratory muscle training might reduce respiratory complications after stroke, but definitive empirical evidence is lacking at the moment.

On 2016 Sep 02, monique dontenwill commented:

In this publication it is not clear if the authors address the ITGA5 integrin or the ITGAV integrin which are quite different integrins. Alpha5 integrin makes an heterodimer with beta1 integrin to form the fibronectin receptor alpha5beta1 and alphav integrin associates with beta3 integrin to form the alphavbeta3 integrin which recognizes fibronectin and vitronectin. Although in the results part it is question of alpha5 integrin,in the discussion only references about alphavbeta3 integrin are given.

On 2016 Nov 28, Lydia Maniatis commented:

Please see comments on PubPeer.

On 2016 Aug 01, Joaquim Radua commented:

Re: the previous comment, I think there may be some unfortunate confusion. Raw p-values of current voxelwise meta-analyses have not the same meaning as usual p-values because they are not derived from the usual null hypothesis (“there are no differences between groups”), but from another null hypothesis (“all voxels show the same difference between groups”). Thus, up to the moment one of the only ways to "approximately" know if the results of a voxelwise meta-analysis are neither too liberal nor too conservative is to compare them with the results of a mega-analysis of the same data, and that's what it was done.

On 2016 Jul 20, Christopher Tench commented:

Here the authors have swapped a method that employs disciplined control of the type 1 error rate (FDR) for a method (SDM) that employs no disciplined control in order to demonstrate there are regions of true positive atrophy in narcolepsy. It does not, however, provide any evidence of this. Arbitrary p-value thresholds do not work in neuroscience. They most certainly do not make a meta-analysis, which demands rigorous statistics. There is no evidence of consistent reporting of regional GM atrophy from this study,

On 2016 Jul 21, Jacob H. Hanna commented:

In this elegant review, Wu and Belmonte discuss in detail (Figure 2 and page 1576 in this manuscript) experiments conducted by Theunissen et al. Cell Stem Cell 2014 Theunissen TW, 2014 claiming complete failure to detect human naïve PSC derived cell integration in chimeric mouse embryos obtained following microinjection into mouse blastocysts, as was reported for the first time by our group (Gafni et al. Nature 2013). However, in this review the authors failed to discuss that among different caveats, imaging and cell detection methods applied by Theunissen et al. Cell Stem Cell 2014 were (and still) not at par with those applied by Gafni et al. Nature 2013.

Regardless, we find it important to alert the readers that Theunissen and Jaenisch have now revised (de facto, retracted) their previous negative results, and are able to detect naïve human PSC derived cells in mouse embryos at more than 0.5-2% of embryos obtained (Theunissen et al. Cell Stem Cell 2016 - Figure 7) Theunissen TW, 2016 < http://www.cell.com/cell-stem-cell/fulltext/S1934-5909(16)30161-8 >. They now also apply GFP and RFP flourescence detection and genomic PCR based assays, which were applied by the same group to elegantly claim contribution of human neural crest cells into mouse embryos (albeit at low efficiency (Cohen et al. PNAS 2016 Cohen MA, 2016).

While the authors of the latter recent paper avoided conducting advanced imaging and/or histology sectioning on such obtained embryos, we also note that the 0.5-2% reported efficiency is remarkable considering that the 5i/LA (or 4i/LA) naïve human cells used lack epigenetic imprinting (due to aberrant loss of DNMT1 protein that is not seen in mouse naive ESCs!! http://imgur.com/M6FeaTs ) and are chromosomally abnormal. The latter features are well known inhibitors for chimera formation even when attempting to conduct same species chimera assay with mouse naïve PSCs.

Jacob (Yaqub) Hanna M.D. Ph.D.

Department of Molecular Genetics (Mayer Bldg. Rm.005)

Weizmann Institute of Science | 234 Herzl St, Rehovot 7610001, Israel

Email: jacob.hanna at weizmann.ac.il

Lab website: http://hannalabweb.weizmann.ac.il/

Twitter: @Jacob_Hanna

On 2016 Jul 14, Jacob H. Hanna commented:

None

On 2016 Aug 11, David C. Norris commented:

This Notice addresses "incorrect confidence intervals and a P value in 2 tables", errors that "were discovered in the course of rechecking the [statistical analysis] code in conjunction with a secondary analysis." My understanding (based on a 1/6/2016 personal communication from a member of the Kessler team) is that said "secondary analysis" would in fact be a reanalysis by myself and Andrew Wilson, which at the time of this Notice was published on F1000Research, and has since completed post-publication peer review and PubMed indexing.¹

Although the Kessler team's discovery of these programming errors does illustrate one benefit of the increased scrutiny a reanalysis engenders generally, it should be understood that this Retraction and Replacement is merely incidental to our reanalysis. In particular, this Retraction and Replacement does not address the original work's² nontransparent application of a dubious PTSD outcome imputation procedure which our reanalysis exposed. It is hoped that some comment on this substantive question will be forthcoming from the original authors, either here on PubMed Commons or through any of several mechanisms^3,4 provided by the F1000Research platform.

1] Norris DC, 2016

2] Kessler RC, 2014

3] F1000Research Policy for Comments on Articles

4] F1000Research - Preparing a Correspondence Article

On 2016 Jun 25, John D. Scott commented:

Response to Moore et al.

John D. Scott, B.Sc. (Agr.), M.Sc.

June 23, 2016

Re: Moore A, Nelson C, Molins C, Mead P, Schriefer M (2016) Current guidelines, common clinical pitfalls, and future direction for laboratory diagnosis of Lyme disease, United States. Emerg Infect Dis 22. doi 10.3201/eid2207.151694

The two-tiered Lyme disease serology test has failed countless patients, including my wife and me. We both have definitive proof of Lyme disease, and yet we have never tested positive using conventional two-tiered Lyme serology testing. We are culture-positive, PCR-positive, and show positivity for anti-Borrelia burgdorferi fluorescent stain on body fluids.

Using circular reasoning to tout the two-tiered Lyme disease serology test is unscientific. This spurious methodology is nothing more than stacking the deck before the experiment begins. Using only seropositive patients to show seropositivity is anti-science and highly flawed. In reality, the two-tiered system fails Lyme disease patients time and time again. When researchers flaunt a screening test that seldom works, it puts the populace at risk.

On 2016 Jun 18, Raphael Stricker commented:

Circular Reasoning in CDC Lyme Disease Test Review

Raphael B. Stricker, MD; Lorraine Johnson, JD, MBA

Previous studies have shown that commercial two-tier serological testing has a sensitivity of about 46% in later-stage Lyme disease in the USA [1]. Commercial two-tier Lyme testing in Europe demonstrates the same poor test sensitivity [2]. The Table in the latest Centers for Disease Control and Prevention (CDC) review by Moore et al. cites three studies allegedly showing that two-tier Lyme testing in later-stage (“non-cutaneous”) Lyme disease has a sensitivity of 87-96% [3]. These numbers will undoubtedly be used to support two-tier testing as a valid diagnostic tool for Lyme disease. Therefore it is important to understand the circular reasoning that produced these inflated and misleading numbers.

Analysis of the three studies cited in the CDC review reveals the following:

+1. Branda et al [4]: Two-tier Lyme test sensitivity 87% (55 patients). The Methods section of this article contains the following language: "All patients categorized as having Lyme disease met the CDC surveillance criteria for the diagnosis.” The reference for this statement [5] contains the CDC surveillance criteria for the diagnosis of Lyme disease. The portion of the CDC surveillance criteria relevant for later Lyme disease is set forth below.

Clinical case definition:

1. Erythema migrans, or
2. At least one late manifestation, as defined below, and laboratory confirmation of infection (emphasis added).

Laboratory criteria for diagnosis:

1. Isolation of Borrelia burgdorferi from clinical specimen, or
2. Demonstration of diagnostic levels of IgM and IgG antibodies to the spirochete in serum or CSF, or
3. Significant change in IgM or IgG antibody response to B. burgdorferi in paired acute- and convalescent-phase serum samples.

This surveillance case definition was developed for national reporting of Lyme disease; it is NOT appropriate for clinical diagnosis (emphasis added).

Comment: Although the Branda et al. study does not say how many later-stage Lyme patients were culture-positive, presumably most were included based on positive serology. Patients who had positive serology as part of the study entry criteria would be expected to have positive serology on the same outcome measure. Circular reasoning.

+2. Molins et al [6]: Two-tier Lyme test sensitivity 96% (46 patients). The Methods section of this article contains the following language: "Lyme disease serology or results from two-tiered testing did not play a role in patient enrollment except for inclusion of late-stage Lyme arthritis patients (emphasis added).”

Comment: Once again, patients with later-stage Lyme disease had to have positive serology in order to be included in the study, and then they had positive serology. Circular reasoning.

+3. Wormser et al [7]: Two-tier Lyme test sensitivity 94% (142 patients). This study was a cost analysis article based on another study [8] that contains the following language: "Lyme arthritis was defined as the presence of joint swelling that was clinically compatible with Lyme arthritis in conjunction with serologic evidence of borrelial infection demonstrated by at least a positive WCS ELISA (emphasis added).”

Comment: Once again, patients with later-stage Lyme disease had to have positive serology in order to be included in the study, and then they had positive serology. Circular reasoning.

Conclusions: Based on circular reasoning, the latest CDC analysis perpetuates the myth that two-tier testing is sensitive for later-stage Lyme disease. The comment in the CDC surveillance guidelines that this testing is NOT appropriate for clinical diagnosis is being ignored by the CDC.

References

[1] Stricker RB, Johnson L. Lyme disease diagnosis and treatment: Lessons from the AIDS epidemic. Minerva Med. 2010;101:419–25.

[2] Ang CW, Notermans DW, Hommes M, Simoons-Smit AM, Herremans T. Large differences between test strategies for the detection of anti-Borrelia antibodies are revealed by comparing eight ELISAs and five immunoblots. Eur J Clin Microbiol Infect Dis. 2011;30:1027-32.

[3] Moore A, Nelson C, Molins C, Mead P, Schriefer M. Current guidelines, common clinical pitfalls, and future directions for laboratory diagnosis of Lyme disease, United States. Emerg Infect Dis. 2016 Jul;22(7). doi: 10.3201/eid2207.151694.

[4] Branda JA, Linskey K, Kim YA, Steere AC, Ferraro MJ. Two-tiered antibody testing for Lyme disease with use of 2 enzyme immunoassays, a whole-cell sonicate enzyme immunoassay fol¬lowed by a VlsE C6 peptide enzyme immunoassay. Clin Infect Dis. 2011;53:541–7.

[5] Wharton M, Chorba TL, Vogt RL, Morse DL, Buehler JW. Case definitions for public health surveillance. MMWR Recomm Rep 1990; 39:1–43.

[6] Molins CR, Sexton C, Young JW, Ashton LV, Pappert R, Beard CB, et al. Collection and characterization of samples for establishment of a serum repository for Lyme disease diagnostic test development and evaluation. J Clin Microbiol. 2014;52:3755–62.

[7] Wormser GP, Levin A, Soman S, Adenikinju O, Longo MV, Branda JA. Comparative cost-effectiveness of two-tiered testing strategies for serodiagnosis of Lyme disease with noncutaneous manifestations. J Clin Microbiol. 2013;51:4045–9.

[8] Wormser GP, Schriefer M, Aguero-Rosenfeld ME, Levin A, Steere AC, Nadelman RB, et al. Single-tier testing with the C6 peptide ELISA kit compared with two-tier testing for Lyme disease. Diagn Microbiol Infect Dis. 2013;75:9–15.

Disclosure: RBS and LJ are members of the International Lyme and Associated Diseases Society (ILADS) and directors of LymeDisease.org. They have no financial or other conflicts to declare.

On 2016 Jun 18, Dirk Lachenmeier commented:

Thank you for the interesting report! Regarding the compounds responsible for allergic contact dermatitis, I believe that wormwood does not contain alcohol per se (this is probably a mismatch with the alcoholic beverage absinthe, which may contain Artemisia absinthium as ingredient). However, wormwood as many other Asteraceae plants contains sequiterpene lactones, which are known to cause such reactions from other species in the family, see e.g. https://www.ncbi.nlm.nih.gov/pubmed/12793186.

On 2016 Jul 07, GianCarlo Panzica commented:

More comments are in a previous article Slama R, 2016

and in a letter submitted after the publication of the decision of the European Commission Kortenkamp A, 2016

On 2016 Jul 22, Christopher Tench commented:

Excellent. The issue with the bug was that there was no FWER control, so without the test you have performed there is no way to know if there are any truly positive results. But now you have confirmed your study. Thanks.

On 2016 Jun 24, Maddalena Boccia commented:

We thank Christopher Tench for his comment following which we have re-run the analysis with GingerALE 2.3.6 where bug in Cluster level inference has been fixed. We are sorry for this but the paper has been accepted before the recognition of the so-mentioned bug. Anyway, we are glad to announce that the results are not changed at all, demonstrating that there is no actual false positive results but just the risk of false positive results, which is, together with the risk of false negative results, the two unavoidable scientific risks.

On 2016 Jun 19, Christopher Tench commented:

The version of GingerALE used (2.3.5) has an implementation issue that produces false positive results. This was fixed at 2.3.6.

On 2017 Aug 24, Serina Stretton commented:

Choi et al’s commentary, rightly condemns the practice of ghost writing in the peer-reviewed literature. Ghost writing (undisclosed contributions from individuals who DO NOT meet authorship criteria) and ghost authoring (undisclosed contributions from individuals who DO meet authorship criteria) are unethical practices and should be eradicated [1,2]. Here we address several of Choi et al’s assertions about the roles and responsibilities of professional medical writers (PMWs) and authors, and provide evidence of how PMWs can help deliver timely and accurate dissemination of clinical trial data.

As acknowledged by Choi et al and multiple editor organizations, including the International Committee for Medical Journal Editors (ICMJE) [1], the Council for Scientific Editors (CSE) [3], and the World Association of Medical Editors (WAME) [4], PMWs (individuals who do not meet authorship criteria and who declare their involvement in the acknowledgements) have a legitimate and valuable role in assisting authors disclose findings from clinical trials in the peer-reviewed literature. Publications involving PMWs are of higher quality compared with publications not involving PMWs or compared with publications that are not funded by industry; they can be more rapidly accepted through the peer-review process [5], more consistently meet the requirements set by international reporting guidelines [6, 7], contain significantly fewer non-prespecified outcomes [8], and are less likely to be retracted due to misconduct [9]. These outcomes are the result of authors working with PMWs who receive mandatory training on ethical publication practices and international reporting requirements from their employers and industry funders [10-12].

Choi et al put forward the scenario whereby authors play a seemingly passive role in the development of peer-reviewed manuscripts funded by the pharmaceutical industry. At worst, Choi et al assert that authors do not have access to raw data, may never have seen their publication before submission, and that industry-funded publications involving medical writers are riddled with embedded marketing messages. These assertions appear to absolve authors from any responsibility or accountability that they have as authors [1, 13]. Earlier this year, the American Medical Writers Association (AMWA), the European Medical Writers Association (EMWA), and the International Society for Medical Publication Professionals (ISMPP) released a joint position statement outlining the respective roles of authors and PMWs [14]. As required by the ICMJE [1] and upheld by the AMWA-EMWA-ISMPP joint position statement, authors must provide early intellectual input to a publication, be involved in the drafting, approve the final version for publication, and agree to be accountable for all aspects of the work. These last two requirements challenge Choi et al’s assertion that authors have no control over the content of their publications. Indeed, when working with PMWs, the PMW’s roles are to assist authors disclose their findings in a timely, ethical, and accurate manner, and to ensure that authors and sponsors are aware of their obligations, that author contributions during development of the manuscript are documented, and that the writer’s and sponsor’s involvement and funding are disclosed transparently and appropriately [12, 14]. In these ways, the PMW serves as a “gatekeeper” of compliance with widely-accepted standards of authorship.

We caution against conflating “ghost authorship” with “ghost writing”. Choi et al state that despite increased awareness of the need to avoid the unethical practice of ghostwriting, the prevalence remains high. In support of this, the authors cite Wislar et al’s survey of honorary or ghost authorship from 2008 [15], which showed that an individual who merited authorship was excluded from the author byline in 7.9% of articles. However, the prevalence of ghostwriting (an unnamed individual who participated in the writing) in this survey was far lower at 0.2%. In addition, findings from a systematic review of the literature suggest that the reported prevalence of ghostwriting in the medical literature can vary, but is on the decrease [16].

Choi et al finish by calling for a ban on “any manuscript that is discovered to be written by people other than the named authors”. We certainly agree if the assistance provided by other people is not disclosed transparently within the manuscript. However, we disagree that a ban on ethically conducted and appropriately acknowledged PMW assistance is warranted. We strongly urge authors and sponsors to select and work with PMWs on the basis of a proven track record and commitment to ethical and transparent publication practices. In addition, we urge authors to become familiar with reporting guidelines and be aware of, and fully comply with their obligations and roles as authors.

The Global Alliance of Publication Professionals (www.gappteam.org)

Serina Stretton, ProScribe – Envision Pharma Group, Sydney, NSW, Australia; Jackie Marchington, Caudex – McCann Complete Medical Ltd, Oxford, UK; Cindy W. Hamilton Virginia Commonwealth University School of Pharmacy, Richmond; Hamilton House Medical and Scientific Communications, Virginia Beach, VA, USA; Art Gertel, MedSciCom, LLC, Lebanon, NJ, USA; Julia Donnelly, Julia Donnelly Solutions Ltd, Ashbourne, UK.

GAPP is a group of independent individuals who volunteer their time and receive no funding (other than website hosting fees from the International Society for Medical Publication Professionals). All GAPP members have held, or do hold, leadership positions at associations representing professional medical writers (eg, AMWA, EMWA, DIA, ISMPP, ARCS), but do not speak on behalf of those organisations. GAPP members have, or do provide, professional medical writing services to not-for-profit and for-profit clients.

REFERENCES [1] www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html; 2016 [accessed 08.06.17] [2] Gøtzsche PC et al. PLoS Med 2009;6:e23. [3] Council of Science Editors. White Paper on Publication Ethics - Council of Science Editors, http://www.councilscienceeditors.org/resource-library/editorial-policies/white-paper-on-publication-ethics/; [accessed 08.17.17]. [4] World Association of Medical Editors. Policy Statement: Ghost writing initiated by commercial companies, http://www.wame.org/about/policy-statements#Ghost Writing; 2005 [accessed 08.17.17]. [5] Bailey, M. AMWA J 2011;26(4):147-152 [6] Gattrell W et al. BMJ Open. 2016;6:e010329 [7] Jacobs A. Write Stuff 2010;19(3):196-200 [8] Gattrell W et al. ISMPP EU Annual Meeting 2017 [9] Woolley KL et al. Curr Med Res Opin 2011;27(6)1175-82 [10] www.ismpp.org/ismpp-code-of-ethics [accessed 08.06.17] [11] www.amwa.org/page/Code_of_Ethics [accessed 08.06.17] [12] Wager E et al. BMJ Open. 2014;4(4):e004780 [13] Battisti WP et al. Ann Intern Med. 2015;163(6):461-4 [14] www.ismpp.org/assets/docs/Inititives/amwa-emwa-ismpp joint position statement on the role of professional medical writers_january 2017.pdf 2017 [accessed 08.06.17] [15] Wislar JS et al. BMJ 2011;343:d6128.4-7 [16] Stretton S. BMJ Open 2014;4(7): e004777.

On 2016 Aug 01, David Keller commented:

A Hybrid Non-Invasive Colon Cancer Screening Strategy Lowers Cost/Benefit With Medicare Coverage

Many patients prefer to avoid invasive procedures and radiation exposure when screening for colon cancer. Non-invasive fecal testing fits these preferences, and can achieve results nearly as beneficial as colonoscopy every 10 years, according to CISNET (the Cancer Intervention and Surveillance Modeling Network) model outcomes displayed in Figure 3 of the 2016 USPSTF Recommendation Statement [1]. Colonoscopy-based screening yields the highest benefit, with an estimated 270 life-years gained per 1000 persons screened, closely followed by non-invasive Multi-Target FIT-DNA ("ColoGuard") testing every year with 261 life-years gained. The next-best non-invasive no-radiation strategy is annual testing for occult fecal blood using either immunochemical testing (FIT) or high-sensitivity guaiac (HSg-FOBT), which gain 244 and 247 life-years, respectively, followed by FIT-DNA performed once every 3 years, which gains only 226 life-years per thousand screened, about 15% less benefit than gained with annual FIT-DNA screening.

Partly due to the high cost of FIT-DNA testing, up to $649 per test [2], Medicare covers it only once per 3 years [2]. Medicare beneficiaries who are not willing to pay over $1200 out-of-pocket for the additional two non-covered test kits required for annual FIT-DNA screening may wish to consider an alternative strategy not discussed by CISNET: FIT-DNA testing every 3 years, plus annual FIT testing in the off-years. This "hybrid" combination should yield a benefit larger than annual FIT testing alone but smaller than annual FIT-DNA testing, putting it in the range of 244 to 261 life-years. Weighted interpolation using 2/3 times the FIT-alone benefit plus 1/3 times the FIT-DNA benefit yields an estimated benefit of 251 life-years per thousand screened, which is about 91 days of life per person screened.

It is uncertain whether the cost of the two annual FIT tests would be covered by Medicare for beneficiaries during their 3-year waiting period for their next FIT-DNA test. If not, the retail price of two FIT kits is about $30 [3], probably acceptable to most beneficiaries. Harms of hybrid every-3-year FIT-DNA screening plus FIT screening in the off-years should be larger than for annual FIT testing, but less than for annual FIT-DNA testing, because FIT is significantly more specific than FIT-DNA, 96.4% versus 89.8% respectively per test [4]. The harms of fecal screening are complications of colonoscopy triggered by positive screening test results, estimated at 12 GI or CV events for annual FIT-DNA, 10 events for annual FIT, [1], with an interpolated estimate of about 10.7 events for the hybrid strategy (all harms are expressed in events per 1000 persons screened).

References

1: US Preventive Services Task Force. Screening for Colorectal Cancer: US Preventive Services Task Force Recommendation Statement. JAMA. 2016 Jun 21;315(23):2564-75. doi:10.1001/jama.2016.5989. PubMed PMID: 27304597. See Figure 3 on page 2569.

2: ColoGuard website, accessed on 7/2/2016. http://www.cologuardtest.com/what-to-expect-with-cologuard/faq/cologuard-cost-how-much-is-cologuard and, Medicare official website, accessed on 7/2/2016. https://www.medicare.gov/coverage/colorectal-cancer-screenings.html

3: Pinnacle Labs website Fecal Immunochemical Test 2-Pack $29.99, accessed on 7/2/2016. https://www.pblabs.com/collections/fit-fecal-immunochemical-test/products/second-generation-fit-fecal-immunochemical-test-2-pack

4: Imperiale TF, Ransohoff DF, Itzkowitz SH, Turnbull BA, Ross ME; Colorectal Cancer Study Group. Fecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population. N Engl J Med. 2004 Dec 23;351(26):2704-14. PubMed PMID: 15616205.

On 2016 Aug 24, Arturo Martí-Carvajal commented:

Dear authors,

I have read your valuable paper. However, at least for me, it is not clear how was conducted and reported statistical analysis in this crossover RCT. Why not reported SD and coefficient correlation? Why was only reported first phase as this RCT had been a parallel-design? Please, see table 2 of full text paper

I think that trial authors should report these statistical information.

Kind regards

Arturo Martí-Carvajal, MD Venezuela

On 2016 Jun 28, thomas samaras commented:

Certainly the authors have a point that hormonal and immunological characteristics affect the differences in longevity between men and women. However, women also have more complex bodies due to their reproductive role. In earlier years, females had similar or lower life expectancies compared to men. This was likely due to high death rates during birth from infections or birth complications without the benefits of modern medicine. Since many women in the past gave birth to many more children compared to today, they were obviously at high risk of reduced longevity.

There's another explanation for the difference in longevity between males and females. The larger size of males, who have the same same length telomeres at birth, exhibit faster shortening of telomeres with aging due to their bigger bodies and increased cell replication needs during their lifetimes. Maier et al. found shorter people at 90 years of age had longer telomeres and lived longer than taller people. Other studies have found shorter telomeres are related to increased CVD and reduced longevity (Cameron; Salpea). Studies of birds by Barrett some years ago and Ringsby (more recently) found that breeding for smaller size resulted in longer telomeres and reduced mortality.

Promislow and Moore in two different studies found that when male mammals are larger than females in a species, the males have greater mortality. And this increase in mortality correlates with increasing body mass. However, when females are larger within the same species, the females have a higher mortality. So body size differences appear to drive differences in mortality. Rollo observed this when he adjusted for differences in males and female body weight and found that the difference in longevity disappeared. Miller also found that when he compared human males and females of the same height, the difference in longevity was small. Poulain found that males and females in a mountainous village in Sardinia were equally likely of reaching centenarian status. Males averaged about 5'3 when they were young and no doubt somewhat taller than the women. However, the men spent the days in the fields or shepherding while the women stayed home. This paradox may be explained by a lower body weight for the men compared to the women.

Data from the Bulletin of WHO (1992) showed a consistent inverse relationship between male and female height differences and life expectancy. These findings are listed below. The first entry identifies the population, the second shows the percent increase in height for males compared to females and the third entry shows the decrease in life expectancy vs. increase in height.

1. 21 European countries averaged together: 7.3% vs. 7.5%

2. California White males vs. females: 9.0% vs. - 9.1%

3. California Asians males vs. females: 7.8% vs. - 7.7%

4. Califonria Hispanic males vs. females: 8.5% vs. - 9.3%

5. California Black males vs. females: 8.7% vs. - 11.7% (This difference may be due to much higher death rate due to violence among Black males)

6. US veterans (males vs. male): 6.4% vs. - 6.9%

7. Baseball players (male vs. male): 4.5% vs. - 4.4%

More recent data collected, included the US, Japan, and Poland. The results were essentially the same, including a comparison between Finnish basketball players vs. cross-country skiers (data provided by Sarna, et al.). The results were: basketball players were 8% taller and had an 8.7% shorter life expectancy.

On 2016 Jun 18, Christopher Southan commented:

This post provides a PubChem mapping for most of the compounds https://cdsouthan.blogspot.se/2016/06/antimalarial-structures-examined-for.html

On 2016 Jul 19, David Keller commented:

Is an opioid more toxic in extended-release than in immediate-release form, for the same total daily dose?

Ray and colleagues list a number of possible toxicities of opioids, and state that long-acting opioids (LAO's) "are of particular concern because the prolonged drug levels might increase toxicity." [1] This raises the question of whether chronic pain patients experience higher mortality due to the long-acting delivery system, and whether an equal daily dose of the same opioid would be safer if taken in divided doses of the immediate-release form. For example, a study comparing the mortality associated with extended-release oxycodone versus immediate-release oxycodone would answer the question of how much of the mortality associated with long-acting opioids in this study was due to the long-acting delivery of opioids versus the opioids themselves. Unfortunately, this study cannot answer that important question.

Ray et al cite a guideline that recommends long-acting opioids "for patients with frequent or constant pain", but which adds the provision: "Short-acting opioids may be used during the initial dose titration period of long-acting formulations and as rescue medication for episodes of breakthrough pain" [2] Another guideline states: "Short-acting opioids are probably safer for initial therapy [than LAO's] since they have a shorter half-life and may be associated with a lower risk of inadvertent overdose." [3] Both guidelines question the safety of initiating LAO's in opioid-naive pain patients and suggest that short-acting opioids (SAO's), taken only as-needed, might be inherently safer for initiating the treatment of chronic pain, at least until the patient has arrived at a stable opioid dosing regimen and developed some tolerance to its adverse effects.

eTable 6 in the Supplement to this study discloses that 96-97% of both the control group and the LAO group were currently taking any dose of SAO, but it is not possible to determine what percentage had been taking a stable total daily dose of SAO's long enough to have completed titration to efficacy and developed tolerance to harms such as respiratory depression. In other words, how much mortality was caused by premature treatment with LAO's of patients who had not taken SAO's long enough to complete dose titration and develop opioid tolerance?

References

1: Ray WA, Chung CP, Murray KT, Hall K, Stein CM. Prescription of Long-Acting Opioids and Mortality in Patients With Chronic Noncancer Pain. JAMA. 2016 Jun 14;315(22):2415-23. doi: 10.1001/jama.2016.7789. PubMed PMID: 27299617.

2: McCarberg BH, Barkin RL. Long-acting opioids for chronic pain: pharmacotherapeutic opportunities to enhance compliance, quality of life, and analgesia. Am J Ther. 2001 May-Jun;8(3):181-6. Review. PubMed PMID: 11344385.

3: Chou R, Fanciullo GJ, et al. American Pain Society-American Academy of Pain Medicine Opioids Guidelines Panel. Clinical guidelines for the use of chronic opioid therapy in chronic noncancer pain. J Pain. 2009 Feb;10(2):113-30. doi: 10.1016/j.jpain.2008.10.008. PubMed PMID: 19187889; PubMed Central PMCID: PMC4043401.

On 2016 Jun 17, David Keller commented:

"Long-Acting Opioids Increase Mortality in Patients With Chronic Noncancer Pain" - erroneous Practice Update headline

Through no fault of the investigators of this trial, the headline over the Practice Update summary of this study mistakes the association demonstrated in this observational study for causality, which can only be proved by means of a prospective, randomized, head-to-head interventional trial comparing long-acting opioids with other options for treating chronic non-cancer pain. Until such results are available, the headline should read:

Long-Acting Opioids Are Associated With Increased Mortality in Patients With Chronic Noncancer Pain

This distinction is important, and is a frequent cause of confusion in writers of headlines about clinical trials. Because serious therapeutic mistakes result from over-valuing observational data, it is important to correct these erroneous headlines. Here is the link to Practice Update, accessed on 6/17/2016, containing the erroneous headline:

http://www.practiceupdate.com/content/long-acting-opioids-increase-mortality-in-patients-with-chronic-noncancer-pain/40326/55/6/1#commentarea

The primary-care expert who discusses this study for Practice Update is Peter Lin MD,CCFP, who writes: "Long acting opioids increase death? This is an important question but one that we can’t ethically answer with a study. Imagine getting consent for this study? We are trying to see if these medications would kill you. So we could not ethically randomize patients to long acting opioids versus antiepileptic or antidepressant treatment and see who dies faster." This comment misses the most important question raised, but not answered, in this paper.

The authors list a number of possible toxicities of opioids, and state that long-acting opioids (LAO's) "are of particular concern because the prolonged drug levels might increase toxicity." This raises the question of whether chronic pain patients experience higher mortality due to the long-acting delivery system itself, and whether an equal daily dose of the same opioid would be safer if taken in divided doses of the immediate-release form. For example, a study comparing the mortality associated with extended-release oxycodone versus immediate-release oxycodone would answer the question of how much of the mortality associated with long-acting opioids in this study was due to the long-acting delivery of opioids versus the opioids themselves. Only by comparing an intrinsically short-acting opioid, such as oxycodone, with its extended-release form, in a head-to-head randomized study, can we isolate and quantify any increased harm of the extended-release delivery system itself. Such a trial would be ethical if the subjects were pain patients who are stable on a short-acting opioid and have been designated as appropriate candidates to switch to the extended-release form. The control group would delay that change for a month, while the intervention group would switch to extended-release immediately. Any difference in harms between these two groups during the first month could then be definitely attributed to the extended-release delivery system itself, because all other variables would be held constant, including the pain medication molecule.

Only a randomized, controlled, head-to-head trial, such as proposed above, can quantify the harms and benefits caused by the extended-release delivery of an opioid molecule as compared with the same total daily dose of the immediate-release form of the same opioid molecule. Answering this fundamental question is necessary before further comparisons can be interpreted, such as between opioid pain medications with differing intrinsic durations of effect, or comparisons between opioids and nonopioid pain treatments.

On 2016 Aug 16, Michelle Kraft commented:

Unfortunately I find this a very limited study. There are several point of care tools that should have been consulted in addition to UpToDate if the authors wanted a comprehensive understanding of the clinical impact of medical journals in point of care tools.<br> Additionally there are several other methods for studying reach and possible impact of journal articles such as altmetrics. A more robust article featuring several point of care products that included different types of altmetrics, while not perfect, would have given a much better picture of clinical impact of medical journals compared to the traditional impact factor.

On 2017 Apr 07, Janelia Neural Circuit Computation Journal Club commented:

Highlight/Summary Wilson and colleagues measured the orientation selectivity of dendritic spines on layer 2/3 pyramidal neurons in ferret visual cortex using two-photon calcium imaging. They are trying to address the question of how tuned output is produced by neurons that receive diverse types of input, a core question in neural computation. They suggest that higher orientation selectivity of individual neurons was correlated with greater clustering of similarly tuned spines (and not a narrower distribution spine preference or residence in a low-rate-of-change location of the orientation preference map). They conclude that dendritic nonlinearities play a critical role in shaping orientation selectivity.

Impact / strengths This is the first study of the functional organization of synaptic input onto the dendrites in a cortical region with columnar architecture. Previous work in mouse visual cortex (Jia et al., 2010) -- which lacks a columnar organization of orientation -- found no evidence of spatial clustering of input orientation preference on the dendrites of layer 2/3 neurons. On a technical level, this work is among only a handful of studies that have used genetically-encoded calcium indicators to estimate the functional selectivity of input to individual spines (cf Chen et al 2013).

By combining two-photon imaging of the dendritic arbor with intrinsic imaging of the orientation preference map, an impressive amount of information was gathered for each neuron. Although the number of neurons was modest (N=9), the number of identifiable spines imaged per neuron (N~300) is respectable for addressing questions of clustering. Signal-to-noise ratio at individual spines was also relatively high, comparable to previous studies (Chen et al., 2013).

Weaknesses The conclusions of this paper are critically dependent on the estimation of calcium influx through synaptic NMDA-Rs and contributions of local-dendritic calcium signals, as well as their interactions. Dendritic electrical events, including local synaptic potentials, local dendritic spikes, and depolarization produced by more global back-propagation of action potentials, cause dendritic calcium influx through voltage gated calcium channels (VGCCs) and also modulate NMDA-Rs. Depolarization interacts non-linearly with NMDA-R Ca influx. The field has not yet converged on methods to disambiguate global and local contributions to spine calcium signals. This is a critical issue for the key claim of the paper (‘clustering of similarly tuned spines’). For example, do the apparent clusters of similarly tuned spines drive the cell (the interpretation that is offered)? Or does the postsynaptic response at an optimal stimulus for the cell potentiate NMDA-R Ca influx in spines to produce an apparent cluster?

Wilson and colleagues estimate the spine signal by subtracting a linear fit of the spine fluorescence against the dendritic fluorescence. They then apply four inclusion criteria (pg 1004 and Methods). After this procedure, spines with orientation preference similar to the parent soma have similar tuning to spines with orthogonal orientation preference; this they take as an indication that the subtraction was effective. Given that some of the inclusion criteria are related to tuning curves (e.g., tuning must be well-described by a Gaussian fit) it was not clear if the inclusion criteria themselves biased this analysis. We would have welcomed analyses of the robustness of the results to changes in the parameters of the subtraction procedures and inclusion criteria.

A related concern is about the spatial scale of the dendritic hotspots. The size of the hotspots is not much smaller than the field-of-view of the images. The inclusion criteria (e.g., a Gaussian fit must explain 70% of the variance) have a possibly strong but unevaluated effect on the results.

On 2016 Sep 27, David Nunan commented:

Important methodological flaws limit the findings from the paper by Ravnskov and colleagues

We performed a post-publication critical appraisal of this paper and found a number of methodological flaws not least: 1. Lack of a published protocol 2. Searching of only one database 3. Nonuniform application of inclusion/exclusion criteria 4. A lack of critical appraisal of the methods used in the included studies 5. No indication of the quality or uncertainty of the included data 6. Issues with the accuracy of data extraction, and, 7. A lack of controlling for confounding due to the effect of lipid-lowering treatment and HDL-C levels presenting major bias and more likely underpinning the majority of the observed inverse associations.

Based on the above identified flaws in the paper by Ravnskov and colleagues we concluded: "Given that the authors failed to account for significant confounding as well as the methodological weaknesses of both the review and its included studies, the results of this review have limited validity and should be interpreted with caution. At this time it would not be responsible, or evidence-based, for policy decisions to be made based on the results of this study".

Our full appraisal can be found on our website here: http://www.cebm.net/cebm-response-lack-association-inverse-association-low-density-lipoprotein-cholesterol-mortality-elderly-systematic-review-post-publication-pee/

On 2016 Jun 16, David Keller commented:

In the elderly, statins reduce heart attacks, strokes and, probably, mortality

Ravnskov and colleagues conclude that their analysis of observational data requires "re-evaluation of guidelines recommending pharmacological reduction of LDL-C in the elderly as a component of cardiovascular disease prevention strategies." [1] I disagree, because the guidelines they challenge were based on large, randomized, placebo-controlled, double-blinded prospective interventional trials, higher-quality studies yielding more convincing data than the observational studies examined by Ravnskov. Statisticians warn us that observational data can only demonstrate associations, not causality, and should only be used for hypothesis generation, not for treatment decisions.[2]

A meta-analysis of eight high-quality controlled trials, including over 24,000 subjects with average age 73 years, was performed by Savarese and colleagues in 2013, which proved that elderly patients with CV risk factors but without established cardiovascular (CV) disease actually do benefit from statin therapy. [3] Statin therapy significantly reduced heart attacks by over 39%, and reduced strokes by over 23%, and non-significantly reduced all-cause mortality by 5.9%, and CV mortality by 9.3%. Mortality trends did not reach significance due, in part, to early termination of studies required by the significant reductions in MI and stroke, which are the #1 and #3 causes of death for the elderly. If these studies could have ethically been continued, the trends in CV and all-cause mortality could only have strengthened.

Elderly patients with CV risk factors do benefit from pharmacologic reduction of LDL-C by suffering fewer heart attacks, strokes and probably reduced mortality. Seniors should not discontinue statin therapy due to this study, which is based on lower quality data than the treatment guidelines are based on.

This comment has been published as an online letter by BMJ Open. [4]

References

1: Ravnskov U, et al. Lack of an association or an inverse association between low-density-lipoprotein cholesterol and mortality in the elderly: a systematic review. BMJ Open. 2016 Jun 12;6(6):e010401. doi:10.1136/bmjopen-2015-010401. PubMed PMID: 27292972.

2: Hannan EL. Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations. JACC Cardiovasc Interv. 2008 Jun;1(3):211-7. doi:0.1016/j.jcin.2008.01.008. Review. PubMed PMID: 19463302.

3: Savarese G, et al. Benefits of statins in elderly subjects without established cardiovascular disease: a meta-analysis. J Am Coll Cardiol. 2013 Dec 3;62(22):2090-9.doi:10.1016/j.jacc.2013.07.069. Epub 2013 Aug 28. PubMed PMID: 23954343.

4: Keller DL, Statins do prevent heart attacks and strokes in the elderly. BMJ Open, published online on June 21, 2016 at the following URL: http://bmjopen.bmj.com/content/6/6/e010401.long/reply#bmjopen_el_9817

On 2016 Jun 15, David Keller commented:

The data in the included studies were all "corrected" under the assumption that LDL-C is a harmful risk factor

This analysis of multiple observational cohort studies reports no association, or an inverse association, between LDL-cholesterol levels and death rates, thereby challenging the widely-accepted hypothesis that high levels of LDL-C are a causal risk factor for cardiovascular atherosclerotic disease (CAD).

However, in table 2, we see that the study by Nilsson and colleagues was "corrected" for non-HDL-cholesterol, a variable which is highly correlated with LDL-C. The effect of correcting for non-HDL-C is almost the same as the effect of correcting for LDL-C itself, as we can see by using the Friedewald Equation to derive non-HDL cholesterol as LDL + Triglycerides/5 [1]

It appears that the authors of all of the analyzed cohort studies corrected their data under the assumption that higher LDL-C levels are harmful. Because Ravnskov is testing that very assumption, any "corrections" made to the original LDL-C data will actually further confound his findings. I suggest that Ravnskov reanalyze the original cohort data without any corrections or assumptions applied at all.

Reference

1: Martin SS, Blaha MJ et al, Friedewald-estimated versus directly measured low-density lipoprotein cholesterol and treatment implications. J Am Coll Cardiol. 2013 Aug 20;62(8):732-9. PubMed PMID: 23524048.

On 2016 Jun 22, Zvi Herzig commented:

The authors mistakenly exclude the possibility that increased gene suppression in e-cigarette users relates to nicotine:

Based on the similar serum cotinine levels in cigarette smokers and e-cigarette users, it does not appear that the overall difference in number and level of gene expression changes is dependent on nicotine.

However, unlike smoking, e-cigarette use frequently entails nasal exhalation (as online videos on the subject can demonstrate). It is already known that nicotine alone affects gene expression Zhang S, 2001, Bavarva JH, 2013. Nasal mucosae of e-cigarette users are exposed to much more nicotine, hence the increased gene suppression.

The health effects of nicotine exposure alone can be established from long-term human studies of snus and NRT use Murray RP, 1996, Lee PN, 2013, Royal College of Physicians, 2016.

On 2016 Jun 14, James Tsung commented:

Battlefield Acupuncture ASP Needle Insertion Video: https://youtu.be/xeEmX3jkvcE

Case 3 Pain from Carpel Tunnel Syndrome in 19 y.o. Female Video: https://youtu.be/oOujSLjcTFI

Case 4 Pain from Appendicitis in 9 y.o. Boy with Morphine Allergy Video: https://youtu.be/OlkJ2f1PP0I

A safer alternative to opioid analgesia by activating the placebo effect or more? More research to follow.

On 2016 Jul 28, Isabelle Boutron commented:

We would like to thank the Hilda Bastian for her interest in our work. We fully agree that our systematic review has some limitations and we acknowledged most of them in the paper. We also fully agree that the peer review system is a complex system and that we need different approaches to explore this system and that other study designs are also important to tackle this issue. We focused on randomised controlled trials as it provides a high level of evidence and one important result of this systematic review is the appalling lack of randomised controlled trials in this field. Despite huge human and financial investments in the peer review process, its essential role in biomedical research, only 7 RCTs have been published over the last 10 years. Yet, the conduct of randomised controlled trials in this field does not raise any important ethical or methodological concern. These results should be a call for action for editors to facilitate the conduct research in this field and give access to their data.

On 2016 Jun 12, Hilda Bastian commented:

This is a helpful broad brush update on randomized controlled trials (RCTs) of peer review interventions in biomedical journals (see older review Jefferson T, 2007 and my comment on that review). However, while the authors list several limitations, including restricting to RCTs and to biomedical journals, there are other limitations that, in turn, highlight the impact of those limitations.

One of those is the outcomes addressed here. The focus is explicitly on the peer reviews themselves and the process, and not wider outcomes, such as potential benefits and harms to peer reviewers or the impact of policies such as open review on journals (e.g. level of unwillingness to review).

In particular, the issue of harms brings us back to the limitation of looking only at RCTs, and limiting to the biomedical literature and a limited scope for databases searched. The authors provide no rationale for limiting the review to biomedical publications. Given that there are so few eligible studies within the scope this review, moving past this is essential. (In a blog post on anonymity, openness, and blinding of peer review in March 2015, in addition to the 11 RCTs identified in this systematic review, I identified a further 6 comparative studies, as well as other types of studies relevant to the questions around which known concerns exist.)

Peer reviewers are not just a means to an end: biases of peer reviewers can have a major impact on the careers of others, and at a minimum, specifically addressing gender, seniority, and institutional/country/language impact is critical to further work on this topic. A more contextual approach is needed to grapple with the complex ecosystem involved here.

A final point that is less likely to have had an impact, but is worth consideration by others working on this issue. Limiting the search strategy to the single term "peer review" may have an impact on searches, as terms such as open review and post-publication review become more widely used. Terms such as manuscript and editorial review, and peer reviewers could also be considered in constructing a search strategy in this area. (To identify the studies to which I refer above, Google Scholar and a wider range of search terms was necessary.)

(Disclosure: Part of my job includes working on PubMed Commons, which does not allow anonymous commenting.)

On 2016 Jun 27, Jafar Kolahi commented:

Editor’s note on this article is available via: http://www.nature.com/bdj/journal/v220/n11/full/sj.bdj.2016.398.html

On 2016 Jun 17, Jafar Kolahi commented:

Full list of Altmetric to 50 Dental Articles-2014 is available via:

http://www.nature.com/bdj/journal/v220/n11/extref/sj.bdj.2016.411-s1.pdf

On 2016 Jun 13, Christopher Tench commented:

The version of GingerALE employed had a bug that results in no control of the false positive results. This was fixed in version 2.3.6

On 2016 Aug 31, Clive Bates commented:

May I suggest that readers first read Professor Polosa's review below, and then turn to Benowitz NL, 2016 for a more credible and complete account of the cardiovascular effects of nicotine as they relate to e-cigarettes. Benowitz and Burbank review the relevant evidence and summarise the current state of knowledge as follows:

The cardiovascular safety of nicotine is an important question in the current debate on the benefits vs. risks of electronic cigarettes and related public health policy. Nicotine exerts pharmacologic effects that could contribute to acute cardiovascular events and accelerated atherogenesis experienced by cigarette smokers. Studies of nicotine medications and smokeless tobacco indicate that the risks of nicotine without tobacco combustion products (cigarette smoke) are low compared to cigarette smoking, but are still of concern in people with cardiovascular disease. Electronic cigarettes deliver nicotine without combustion of tobacco and appear to pose low-cardiovascular risk, at least with short-term use, in healthy users.

This should be a basis for reassuring and encouraging smokers considering switching to vaping.

As happens too often in this field, an author makes claims that go well beyond the scope of their scientific study or their field of expertise. Remarks made to media by the lead author at a cardiology conference in Rome caused a news storm in Britain, and will have increased interest in this paper. My examination of the basis for the claims made and reported is available on my blog > here.

On 2016 Aug 31, Riccardo Polosa commented:

The scientific letter lacks of many important details about participants' previous smoking history, type of device and e-liquid used in the study sessions.

Nonetheless, this is essentially an acute study showing what it is already know about the acute effect of nicotine on arterial stiffness. Besides, comparable acute changes in stiffness would also occur after drinking coffee (caffeine is not a risk factor for cardiovascular disease), exercising (exercise is beneficial for cardiovascular health), being exposed to emotional stress, and after taking nicotine replacement therapies.

Most of the observed changes in arterial stiffness after e-cigarette use were induced by asking participants to vape continuously for 30 min (!!!) (and only then the effect was similar to puffing a single conventional cigarette). Obviously, vaping continuously for 30 min is NOT a realistic conditions of use.

Besides, there are already two large clinical studies (one acute and the other chronic) that contradict the negative take of this research letter in relation to cardiovascular health:

http://www.ncbi.nlm.nih.gov/pubmed/24958250

http://www.ncbi.nlm.nih.gov/pubmed/26749533

Last but not least, measurement of arterial stiffness after acute exposure to active stimuli has no prognostic value whatsoever. That is why current guidelines on arterial stiffness measurements clearly state that before the measurements subjects should abstain from the use of any stimulants (like nicotine, caffeine and alcohol) for at least 4-6 hours.

On 2016 Jun 18, Zvi Herzig commented:

Drinking coffee also acutely increases aortic stiffness and blood pressure Vlachopoulos C, 2003. It is thus surprising that these transient effects are suggested as possible predictors of harm.

On 2017 Jun 16, David Keller commented:

To the Editor: The study by Eguale and colleagues [1] demonstrates that off-label use of prescription drugs is associated with increased ADEs (adverse drug events) only when such use lacks strong scientific evidence. Specifically, off-label uses of drugs with strong scientific evidence had the same risk of ADEs as on-label use. This finding implies that the extreme expense and delay caused by the process of U.S. Food and Drug Administration (FDA) approval of an already-approved drug for a new indication may not be necessary if strong scientific evidence supports such use. The take-home message of this study is not that we need to crack down on off-label prescribing, but that we need to crack down on unscientific prescribing.[2] Electronic health records should be programmed to discourage unscientific prescribing, not off-label prescribing. Because off-label prescriptions backed by strong evidence are just as safe as prescriptions for FDA-approved indications, the FDA ban on promotion of the former denies patients the benefits of safe and scientifically proven medications. The focus should shift to suppression of off-label prescribing only when it is not backed by strong evidence.

Conflict of Interest Disclosures: None reported.

References

1: Eguale T, Buckeridge DL, Verma A, et al. Association of off-label drug use and adverse drug events in an adult population. JAMA Intern Med. 2016;176(1):55-63.PubMedArticle

2: Good CB, Gellad WF. Off-label drug use and adverse drug events: turning up the heat on off-label prescribing. JAMA Intern Med. 2016;176(1):63-64.PubMedArticle

On 2017 Feb 28, Zvi Herzig commented:

To conclude that dual use does not increase cessation, plausible alternative explanations of the data must be excluded, including the very plausible possibility that treatment failures are more resistant to cessation compared to ordinary smokers.

On 2016 Jun 13, Lamberto Manzoli commented:

Well, the distinction between quitting failures and treatment failures, overall and especially in our study, is based upon no data and, as such, it is just a hypothesis. In any case, even if such a distinction would have been "significant", it would not invalidate the main point: in real life, dual users did not quit tobacco more frequently than tobacco only smokers.

On 2016 Jun 10, Zvi Herzig commented:

The authors note that "dual use did not improve the likelihood of quitting tobacco or e-cigarette use". However, dual use indicates a history of resistance to treatment (ie, e-cigarette use without cessation). This severe bias restrains meaningful comparison of cessation-rates between dual users and other smokers.

Indeed, the authors note: "It has been suggested that dual users are frequently ‘quitting failures’ and thus biased against cessation. However, in our sample, the proportion of those who tried quitting before was quite similar between tobacco smokers (33.7%) and dual users (35.9%)."

However, the source quoted for this suggestion (McRobbie H, 2014) mentions 'treatment failures' rather than 'quitting failures'. This is significant: Regardless of whether dual users have failed cessation or not, they have shown resistance to EC-induced cessation. Furthermore, if indeed dual use is not associated with attempted cessation, then dual users have demonstrated a special interest in nicotine and disregard for health, which likely biases against cessation.

On 2017 Oct 13, Gerhard Nebe-von-Caron commented:

looks like the paper slipped through the review process somehow. Whilst the journal published an erratum to clarify some volumes used, they did not check for coherence of the methodology.

Ultra Rainbow beads were supplied in a dropper bottle, which was inconvenient for volume control. The dropper lid was part of the bottle design and could not be removed. The volume of a drop (50ul) was specified by the datasheet.

Considering that it would be well known to anyone skilled in the art that the accuracy of counting is dependent on the accuracy of the volume of the materials used as clearly described in the spherotech method datasheet http://www.spherotech.com/Updated STN 8-21-07/STN-15 Rev B.pdf "Procedure To obtain accurate absolute cell counts, the SPHEROTM AccuCount Particles are used in conjunction with flow cytometry. The SPHEROTM AccuCount Particles have a concentration of approximately 1x106 particles/mL. The actual concentration is listed on the Technical Data Sheet for the product. The first step during sample preparation is to add the monoclonal antibody to 100μL of the test sample. The sample is then incubated, lysed, washed, and resuspended in 1 to 2 mL of phosphate buffer saline, 0.1M, pH 7.4. If staining and lysing are not necessary, add a known volume of test sample to the 1 to 2 mL of phosphate buffer saline. Washing the AccuCount Particles with the sample prior to analysis is discommended because a reduction in the number of reference particles will occur. Next, add exactly 50μL of the AccuCount Particle to the suspension. The precision during pipetting of the AccuCount Particles is absolutely critical. The sample is then analyzed by flow cytometry. The bead and cell population are gated on the fluorescence and/or side scatter channel. Record the number of events for the AccuCount Particles and the test sample. The absolute cell count is then determined with the following equation. ..."

Considering that the authors claim that they dispensed the particles by reverse pipetting but then claimed the poor reproducibility of the spherotech beads because they used the drops for direct dispensing is ironic. If I remember right the AccuCount beads are screwtop and the normal Ultra Rainbow beads are usually used for detector linearity check for which the dropper dispensation is perfectly adequate, for absolute counting this is inadequate and the authors should have been aware of that. If they wanted to use the beads they had in the fridge they should have just pre dispensed the approximate volume and then pipetted accurately.

and if Fig 1 confuses you - it's probably because they got the axis labels the wrong way round

On 2016 Jun 24, Ole Jakob Storebø commented:

We think that the clinicians prescribing methylphenidate for ADHD and others have been insufficiently critical of the literature for decades, trusting that the quality of methylphenidate research was reasonable. In accordance, Shaw in an editorial accompanying our JAMA article Storebø OJ, 2016 stated that the Epstein et al. review on methylphenidate for adults with ADHD was an example of good assessments of quality Shaw P, 2016. It seems Shaw erred, as the Epstein et al. review has now been withdrawn from The Cochrane Library due methodological flaws Epstein T, 2016.

Banaschewski et al. suggest that we included five trials in our analyses that should have been excluded. We think they are wrong. They highlight four trials which they cite as having used “active controls” whereas these are actually co-interventions, used in both the methylphenidate and the control group. Such trials are includable in accordance with our protocol Storebø OJ, 2015. Moreover, excluding these trials from our review would only have produced negligible changes in our results. Furthermore, the trial including children aged 3 to 6 years ought also to have be included in accordance with our protocol. Excluding all five trials would not have changed our conclusions at all. We concluded that methylphenidate might improve teacher reported symptoms of ADHD. However, the very low quality of the evidence, the magnitude of that effect size is uncertain. A change in the effect size of 0.12 points on the standardised mean difference of this outcome would not change anything.

In a subgroup analysis comparing parallel trials and crossover trials, we did not find a significant difference either. However, we noted considerable heterogeneity between the two groups of trials. It is not recommended to pool cross-over trials which only have “end-of-trial data” with parallel group trials (http://handbook.cochrane.org/) and had we done so we would have would risked introducing a “unit-of-analysis error” as we only had “end-of-trial data” from these cross-over trials.

We agree that the variability of the minimal relevant difference is important which is why we reported the 95% confidence interval of the transformed mean value in our review Storebø OJ, 2015. Banaschewski et al. also suggest that we have overlooked information in the Coghill 2007 trial and thereby wrongly assessed this as a trial with “high risk of bias”. We stated in our protocol that we would consider trials with one or more unclear or high risk of bias domain as trials with high risk of bias Storebø OJ, 2015.

We did not overlook information from the Coghill 2007 trial but twice emailed the authors for additional information. They did not respond. The information required is not available in the published study. We presented the risk of bias assessments for the various domains of all 185 included trials. It is correct that we assessed seven cross-over trials as low risk of bias and not six as reported. Thank you for spotting this error. The seventh trial is reported, however, in our table in which the risk of bias assessments for all the domains is shown. All trials, irrespective of vested interest bias, were regarded as having a high risk of bias due to broken outcome assessor blinding given the easily recognisable, well-known adverse effects of methylphenidate. When adding this seventh cross-over trial to the subgroup analysis on the outcome “teacher-rated ADHD symptoms – cross-over trials”, we now find significant differences between the trials with “high” compared to “low” risk of bias (standardised mean difference (SMD) -0.96 [95% confidence interval -1.09 to -0.82] compared to -0.64 [-0.91 to -0.38]. Test for subgroup difference: Chi² = 4.27, df = 1 (P = 0.04), I² = 76.6%).

Banaschewski et al. focus only on our assessment of risk of bias and do not mention the core instrument for assessing quality of meta-analyses namely the Grades of Recommendation Assessment Development and Evaluation (GRADE) approach Andrews J, 2013. Our assessment of the evidence as “very low quality” is not only based on the assessment of risk of bias, but also on other factors such as heterogeneity, imprecision, and indirectness of the evidence. This is clearly reported in our review.

We downgraded the quality of the included trials in the meta-analysis for imprecision and for moderate heterogeneity. The durations of included trials were short, with an average of 75 days. Most patients receive methylphenidate treatment for substantially longer periods and the beneficial effects may diminish over time Jensen PS, 2007 Molina BS, 2009. The short trial duration could suggest the need for further downgrading for “indirectness” according to GRADE Andrews J, 2013. We did not downgrade for this, but we could have. This further underlines that the evidence for the benefits and harms for the use of methylphenidate for children and adolescents with ADHD is of very low quality.

We have assessed 71 trials as having high risk of bias in the “vested interest” domain as they were funded by the industry and/or the authors were affiliated with the industry.

It is not incorrect for us to state that none of the trials funded by the pharmaceutical industry showed a low risk of bias in all other areas as we considered all the trials as high risk of bias on the domain of blinding. This is clearly reported in our review.

We have now conducted the requested subgroup analysis comparing those trials with high compared to low risk of vested interest bias on the teacher-rated ADHD symptoms outcome. The effect of methylphenidate in the 14 trials with high risk of vested interest bias was SMD -0.86 [-0.99 to -0.72] compared to SMD -0.50 [-0.69 to -0.31] in the 5 trials with low risk of vested interest bias. Test for subgroup differences is Chi² = 8.67, df = 1, P = 0.003. So even in this small sample we find a significant difference.

We recommend Banaschewski et al. to read the essay by John P Ioannidis about vested interests Ioannidis JP, 2016.

It is important to stress that the results of our review would have been the same had we disregarded the issue of vested interest.

Had there been inconsistencies regarding one domain of bias in a few trials they would not change the fact that these trials are to be considered as trials at high risk of bias. For example, in two trials, Konrad 2004 and Konrad 2005, there is inconsistency in how our author teams assessed the randomisation process. However, both trials have several other domains at “unclear risk of bias” or “high risk of bias”. In the Ullman 2006 trial, three domains are assessed as “unclear risk of bias”. In Wallace 1994, five domains are assessed as being of “unclear risk of bias” and one as “high risk of bias”. In Wallander 1987, five domains are assessed as “unclear risk of bias”. Even if there was inconsistency between one or two items, these trials are high risk of bias trials. There may well be small differences in our judgements, but that does not change the fact that the trials included are, in general, trials at high risks of bias Storebø OJ, 2015. It is important to understand that we followed the Cochrane guidelines in every aspect of our review.

Conclusions

We have demonstrated that the trial selection in our review was not flawed and was undertaken with sufficient scientific justification The effect sizes are not too small. We have followed a sound methodology for assessing risk of bias and our conclusion is not misleading. We are concerned about the state of the academic literature and at the financial and academic waste that has occurred, given that more than 250 reviews and 3000 single works have been published on psychostimulants for ADHD treatment. Despite this, there is still no sound evidence regarding the benefits and harms of methylphenidate.

Ole Jakob Storebø, Morris Zwi, and Christian Gluud.

On 2017 Aug 02, Han-Xiang Deng commented:

We recently reported mutations in TMEM230 in familial Parkinson’s disease (PD). Farrer et al raised the concern that mutations in TMEM230 may not be pathogenic to PD. We seriously evaluated Dr. Farrer’s assertions. We obtained updated clinical information and performed several new experiments, including MegaEx chip screening of the family DNA samples with ~2 million SNPs for whole-genome linkage study and re-analysis of whole-exome sequencing data. We did not find any other locus more robust than the chromosome 20p (TMEM230), nor any other variants with better segregation than TMEM230-R141L to explain the inheritance of PD in the large Mennonite family. Based on the new genetic data from the Mennonite PD family, and the robust genetic data showing additional TMEM230 mutations in multiple PD families, we are confident to conclude that TMEM230 is a new PD-causing gene. Further studies of TMEM230 should provide important mechanistic insights into understanding the vesicle/endosome trafficking/recycling defects in the pathogenesis of PD. A detailed response is available in bioRxiv (http://www.biorxiv.org/content/early/2017/07/31/170852).

On 2016 Dec 31, MATTHEW FARRER commented:

We argue that TMEM230 mutations are neither disease-linked nor impair synaptic vesicle trafficking. Analysis of chromosome 20 STR genotyping, and of nucleotide mutations, in the largest Mennonite kindred shows the work by Deng and colleagues is erroneous. A detailed critique is available in bioRxiv (http://biorxiv.org/content/early/2017/01/01/097030). We recommend further assessment of genotype analysis/original samples (available on request) prior to embarking on TMEM230 biology. We caution that TMEM230 results are unlikely to contribute to a molecular understanding of parkinsonism.

On 2016 Jun 24, Erick H Turner commented:

I am the author of the NEJM article cited as reference #17, in which we showed stark differences in antidepressant efficacy according to two data sources–published journal articles and FDA reviews. The authors of this BMJ Open protocol list a number of sources they plan to search. What is not clear is how what they plan to do when faced with results of the same clinical trial from two (or more) sources. In our NEJM article, we found 11 trials which were positive according to journal articles but negative according to the FDA. When one considers that those journal articles were authored by those with a conflict of interest, and that the discrepancies were due to post hoc outcome switching, it seems clear that FDA reviews should be prioritized as the more credible data source.

On 2016 Jul 30, Chandan Kumar commented:

Kumar-Sinha et al, Transcriptome analysis of HER2 reveals a molecular connection to fatty acid synthesis. Cancer Res. 2003 Jan 1;63(1):132-9. PMID:12517789

On 2016 Jun 17, Timothy M Krahn commented:

It is important to emphasize that our study is about documented disclosure of information that should be provided to patients in accordance with minimum ethics standards for consent to IVF. Most of the critical points made by Hilde Bastian are openly avowed in the limitations section of the paper. We explicitly acknowledged the declining participation rate over time and the fact that fewer of the participating Canadian IVF clinics provided us with information sheets. To be clear, the documents we very specifically requested and the documents we reviewed (as stated in the article) were ‘consent documents’ comprised of both consent forms and accompanying information sheets.

As regards our content analysis, disclosure elements were deemed present if mentioned anywhere in the consent documents reviewed. Mere mention of a keyword (or keywords) for an information element was sufficient for that element to be deemed present as part of documented disclosure. We used this strategy to remove any subjectivity as regards the quality of disclosure, and to be as generous as possible (from a clinic perspective) as regards any conclusions that might be drawn. In practice, this meant that if a consent form indicated that information about a disclosure element was in an accompanying information sheet, that element would have been judged as present just on the basis of the keyword and/or a rudimentary description alone.

Thus, while Bastian is correct that information elements might have been present in the information sheets known to exist but not provided to us, reasonable efforts were made to minimize this possibility. We could do no better given that we could only work with the consent documents that the clinics provided. For the 2014 data set, all clinics were sent an initial request by email followed by a posted letter. If there was no response, those remaining were contacted at least once by telephone. In our estimation, there is nothing more we could have done to generate better and more comprehensive information without increased collaboration by Canadian IVF clinics.

We stand behind our conclusion that “the disclosure of information relevant to the interests of those undergoing IVF and those who are born as a result of IVF appears to be decreasing. Furthermore, the information that increasingly is being disclosed in consent documents appears to be directing the orientation and content of these documents away from the primary interests of the relevant women, couples, and children.” [emphasis added] This general and avowedly tentative conclusion is consistent with the limited representativeness of the data and this we openly acknowledged.

On 2016 Jun 11, Hilda Bastian commented:

This is a vitally important issue to address, but the data presented in this study do not support the authors' conclusion that disclosure of information is decreasing.

The study is characterized by a steeply declining availability of data for comparison - in both the response rate and amount of data provided by those clinics responding. From the first to the last year studied:

• The response rate dropped from 65% to 31%.
• The percentage of consent forms referring to other information sheets as the vehicles for informing women/couples rose from 55% to 100%.
• The percentage of those information sheets available for the study dropped from 82% to 18%.

This suggests that a principal finding, based on a minority of clinics in 2014, is a shift towards providing supplementary information to consent forms, rather than consent forms as the sole formal vehicle of disclosure. If those information sheets were available - and 82% were not in 2014 - the conclusion of this study could be very different.

It would be useful to know if the consent forms indicated that the person signing had been provided with the supplementary information, as part of the formal disclosure. Clarification from the authors would also be useful on whether data from information sheets was included in the results Table, or whether only consent forms themselves were the source.

If both types of consent documents are included, then for 2014, complete consent documents were available for only 2 of 35 clinics (6%), compared with 9 of 17 clinics in 1991 (53%). And the increased coverage of items in the earlier years could be attributable to the enlarged scope of materials assessed.

On 2016 Jun 07, Lily Chu commented:

There is one online self-management resource already available to patients internationally. The website is cfidsselfhelp.org and was founded by Dr. Bruce Campbell, an expert on self-management of chronic diseases, with contributions from Dr. Charles Lapp, a physician specializing in ME/CFS. They also collaborated on another website treatcfsfm.org. Both of these sites contain a plethora of free information/ worksheets patients can use; patients may also join a low-cost online class if that suits their learning better.

As a physician, I am very familiar with the time pressures of clinic and have referred multiple people to these websites with good effect. (I have no financial interest in or relationship to either site.)

On 2016 Jun 22, Christopher Tench commented:

Thanks for the update Xin. The false positives I refer to are in relation to that expected from the stated method. For any method there needs to be a decision made, before the experiment, about what risk of false positive is acceptable. This may be arbitrary, as you say. Nevertheless, the results only meet the accepted risk when the method is implemented correctly. The problem with the implementation prior to 2.3.6 is that it did not control the FWE. As such there was no way to know whether there were any significant results at all until, as you have done, the appropriate checks and updates were performed.

Chris

On 2016 Jun 21, Xin Di commented:

Hi, Chris,

Thank you for your interest in our paper.

The bug you referred to was announced on April 26, 2016 (2.3.6 http://brainmap.org/ale/readme.html). Our data analysis was performed, and our manuscript was submitted before this date. I could not see any possibility that we could use the 2.3.6 version in our paper.

As described in the Brainmap forum, this bug makes cluster-level threshold more lenient. We re-analyzed our data using GingerALE version 2.3.6, and confirmed that some small clusters reported in our analysis were no longer significant at the same threshold of cluster-level p < 0.05. But large clusters for each of the analyses are still significant. The idea of our paper is that there are consistent task modulated connectivity with the amygdala, and different tasks may modulate amygdala connectivity with different brain regions. Our conclusion will not be affected if we used version 2.3.6, because it is not based on any single clusters.

Lastly, your comment that older versions of GingerALE have "a bug that produces false positive results" is only partially correct. Indeed, all statistical methods produce false positive results, not to mention the so-called type II error. The bugged version is more likely to produce false positive results. It doesn't mean that all the results are false positive. The statistical threshold is arbitrary, anyway. Why do we use p < 0.05, but not p < 0.03 or p < 0.080808? My point is, when drawing conclusions from data, we need to consider the pattern of results, but not a specific result from arbitrarily picked threshold.

Best, Xin

On 2016 Jun 13, Christopher Tench commented:

The version of GingerALE used in this paper has a bug that produces false positive results. The bug has recently been fixed in version 2.3.6.

On 2017 Jun 30, Seán Turner commented:

Traore et al. (2015) [PMID 27257486] and Vicino et al. (2016) [PMID 27660714] both claim DSM 29078 to be the type strain of Bacillus andreraoultii and Murdochiella massiliensis, respectively. DSM 29078 is not listed in the DSMZ online catalog of strains as of 30 June 2017.

On 2017 Apr 26, L. Norton commented:

The conclusion is justified. Other studies have shown convincingly that anesthesia acutely raises glucose levels in BL/6J mice in the absence of a glucose load (PMID: 16332272). For some forms of anesthesia, the effect can last for at least 60 minutes. Of course this may differ somewhat between BL/6 strains, but I would expect to see similar results to be honest.

On 2017 Jan 31, Alexander Kraev commented:

The conclusion of this study is not justified, as the authors used only one mouse strain. Also it would be useful to know the correct name for their strain, whether it was C57BL/6J or C57BL/6NCrl. C57BL/6J, which was likely used in this work, is known to have glucose intolerance in the absence of anesthesia due to deletion of the Nnt gene. Hence, the correct conclusion would be that C57BL/6J should not be used in metabolic studies.

On 2017 Jun 12, Daniel Weeks commented:

As Neuron appears to have deleted our original comment, here is a copy for the record.

NR1H3 and multiple sclerosis: questionable assumptions and miscalculated p-values

Wang et al (2016) investigated the role of the nuclear receptor NR1H3 in familial multiple sclerosis (MS), and described two of the highlights of their research as (1) “An arginine to glutamine mutation in NR1H3 causes multiple sclerosis in families” and (2) “Common variants in NR1H3 are associated with primary progressive multiple sclerosis”. Regarding the first claim, in a comment on PubMed Commons (http://www.ncbi.nlm.nih.gov/pubmed/27253448#cm27253448_16159), Eric Vallabh Minikel and Daniel MacArthur raised concerns on the basis of the frequencies of the implicated NR1H3 variant, rs61731956, encoding p.Arg415Gln, in the Exome Aggregation Consortium (ExAC). Minikel and MacArthur point out that “the variant is not significantly enriched in cases over ExAC population controls (P = .56) - indeed, its allele frequency is lower in MS cases (0.02%) than in ExAC European population controls (0.03%)”.

As evidence of co-segregation of rs61731956 with disease, Wang et al (2016) report a maximum LOD score of 2.20 at θ=0. However, this LOD score was computed under a fully penetrant model where “unaffected mutation carriers were treated as having an unknown disease status”. This overstates the evidence for co-segregation because in linkage analysis disease status should be assigned blind to mutation status, and so a reduced penetrance model should have instead been used where unaffected individuals were properly coded as having an unaffected disease status. Indeed, later in their manuscript, Wang et al (2016) cite the presence of “three obligate carriers and an unaffected biological family member” as evidence of incomplete penetrance.

The evidence for the second claim, that “common variants in NR1H3 are associated with primary progressive multiple sclerosis”, is also overstated in Wang et al (2016) because the p-values in their Table 1 were incorrectly computed. The p-values presented were derived by computing a 2 degree of freedom chi-squared statistic based on the 3 x 2 genotype table and then looking up the p-value of that statistic using a 1 degree of freedom distribution. In our Table 1, we present the correct p-values for the 2 degree of freedom chi-squared statistic. But as some of the cell counts are small, it would be more appropriate to use a Fisher’s Exact Test. Using the p-values of a Fisher’s Exact Test, and applying a Bonferroni’s correction for the 15 tests carried out (instead of correcting for only 5 as Wang et al did), none of the findings are significant at the 0.05 level after correction for multiple testing.

Based on the concerns raised by Minikel and MacArthur as well as here, it seems that the original claims were over-stated, and it is likely, based on the data presented, that this variant and gene may play no significant roles in MS. Of course, the collection of additional independent data followed by careful and correct statistical analyses will ultimately clarify whether or not NR1H3 plays a role in MS risk. Indeed the International MS Genetics Consortium has already examined this variant in their data (http://biorxiv.org/content/early/2016/07/01/061366), and found “no evidence that this variant is associated either with MS or disease subtype.”

Simon C. Heath^1,2 and Daniel E. Weeks³

¹ CNAG-CRG, Centre for Genomic Regulation (CRG), Barcelona Institute of Science and Technology (BIST), Baldiri i Reixac 4, 08028 Barcelona, Spain

² Universitat Pompeu Fabra (UPF), Barcelona, Spain

³ Departments of Human Genetics and Biostatistics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15217 USA. weeks@pitt.edu

References

Z. Wang, A.D. Sadovnick, A.L. Traboulsee, J.P. Ross, C.Q. Bernales, M. Encarnacion, I.M. Yee, M. de Lemos, T. Greenwood, J.D. Lee, et al. Neuron, 90 (2016), pp. 948–954

On 2016 Aug 08, Daniel Weeks commented:

We raise additional concerns in a comment posted in the 'Comments' section of Neuron's web page for this paper, accessible via this link: http://www.cell.com/neuron/comments/S0896-6273(16)30126-X. These additional concerns center on questionable assumptions in Wang et al's LOD score computations as well as miscalculated p-values in their Table 1.

Simon Heath^1,2 and Daniel E. Weeks³

¹ CNAG-CRG, Centre for Genomic Regulation (CRG), Barcelona Institute of Science and Technology (BIST), Baldiri i Reixac 4, 08028 Barcelona, Spain

² Universitat Pompeu Fabra (UPF), Barcelona, Spain

³ Departments of Human Genetics and Biostatistics, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15217 USA

On 2016 Oct 21, Carles Vilarino-Guell commented:

Case-Control Studies Are Not Familial Studies (2016) Neuron; Volume 92, Issue 2, p339–341.

http://www.cell.com/neuron/fulltext/S0896-6273(16)30695-X

On 2016 Jun 29, Chris Cotsapas commented:

We sought to validate the association of rs61731956 with MS susceptibility in our ongoing study of low-frequency missense variation in MS. After stringent quality control, we used linear mixed models to meta-analyze 32,852 cases and 36,538 controls of European ancestry in 14 country-level strata, genotyped for 250,000 low-frequency non-synonymous variants across all exons using Illumina's HumanCore Exome array. We detected the minor allele rs61731956-A in nine of our strata, but find no evidence of association with overall MS risk (meta-analysis beta = 0.06, p = 0.32). As Wang et al report this association specifically with PPMS, we compared 1,399 PPMS cases to 13,537 RRMS cases directly in five strata with available clinical course information, and also find no evidence of association with disease subtype (meta-analysis beta = 2.35, p = 0.39). Our previous linkage analysis of >700 multiplex families [Sawcer S, 2005] further supports this conclusion (multipoint LOD = 0.0), as does earlier work from the Canadian Collaborative Project on the Genetic Susceptibility to Multiple Sclerosis (CCPGSMS) with no evidence of linkage in 40 Canadian families with four or more affected individuals, authored by members of the Wang et al study team [Willer CJ, 2007].

Based on their interpretation of segregation patterns for rs61731956, Wang et al go on to genotype common variants in the NR1H3 locus in 2,053 MS patients and 799 healthy controls, but fail to detect any association with overall MS risk. They then report that four of their familial cases have a clinical course consistent with that of primary progressive MS (PPMS), and perform a secondary, stratified analysis of clinical course with the five tagging SNPs. They reduce their sample size to 420 PPMS and 1,287 RRMS patients for whom clinical course information was available, and describe an association between rs2279238 (OR = 1.35, p = 0.001) and PPMS, but not RRMS, risk. The IMSGC has already reported a disease risk association in this region [International Multiple Sclerosis Genetics Consortium (IMSGC)., 2013] based on 14,498 MS cases and 24,091 controls to rs7120737, which is 420kb away and in moderate LD with rs2279238 (r² = 0.62, D'= 0.82 in the 1000 Genomes CEU panel). We also genotyped rs3824866, a perfect proxy for rs2279238 (r² = 1, D' = 1 in the 1000 Genomes CEU panel), which shows modest association to MS risk (p = 2.1 x 10^-5). Conditioning on rs7120737 fully explains this association, indicating that the result reported by Wang et al is a modest proxy for the strong signal we have previously reported.

Our 13-fold larger dataset therefore supports a more conventional interpretation of the data presented by Wang et al: there is no association between the low frequency NR1H3 p.Arg415Gln variant rs61731956 and MS risk, but a common haplotype spanning the NR1H3 locus is associated with overall MS susceptibility, despite the failure of Wang et al to detect it in their modestly sized cohort. Our data does not support an association specific to clinical course or PPMS. The false positive likely arose because Wang et al base their conclusions on a total of four affected carriers of the variant, and contravene standard practice by analyzing only five polymorphisms in the NR1H3 locus, not controlling for population stratification, and failing to meet rigorous thresholds of significance for common variation (p < 5 x 10^-8) or for family-based linkage (LOD > 3) [Kruglyak L, 1995; Altshuler D, 2008].

Beyond these technical issues, Wang et al appear to have succumbed to an error in logic in their analysis. Although individually rare, coding variants are exceedingly common in the population: of 7,404,909 variants identified by the Exome Aggregation Consortium in 60,706 individuals, 99% have a minor allele frequency of <1% and 54% are seen exactly once in those data. Therefore, there is complete certainty of observing at least one such variant in two closely related individuals, as Wang et al have done. This is reinforced by their observation of rs61731956-A in multiple unaffected individuals and the presence of this variant in 21/60,706 unselected ExAC individuals. This is not a unique false positive finding, as previous studies of equivalently small sample size have reported MS risk associations to low-frequency coding variants in CYP27B1 [Ramagopalan SV, 2011] and SAIE [Surolia I, 2010], with both results failing to replicate in much larger studies with adequate statistical power [Hunt KA, 2011; Ban M, 2013; Barizzone N, 2013].

We note that the experimental demonstration that p.Arg415Gln alters the heterodimerization efficiency between the NR1H3 product liver X receptor alpha and the retinoid X receptor alpha has no bearing on the association to MS pathogenesis. Many non-synonymous variants have dramatic effects on protein function, but in the absence of robust association to disease this alone cannot support a pathogenic argument [MacArthur DG, 2014].

The combination of our negative results from a 13-fold larger dataset and the methodological and logical flaws in the work presented by Wang et al categorically refute the bold claim that NR1H3 variation defines a Mendelian subtype of MS, which would be the first monogenic form of the disease ever described. Such a discovery would have enormous implications for diagnosis of a subset of cases, prognosis and genetic counseling of extended family members, and eventually for clinical management of the disease in carriers. Unfortunately, the evidence provided by Wang et al does not support this conclusion.

A more complete version of this response, including data tables, is available on biorxiv.

Chris Cotsapas, Yale School of Medicine and Broad Institute of Harvard and MIT, on behalf of the IMSGC.

On date unavailable, commented:

None

On 2016 Jun 27, Daniel MacArthur commented:

We thank the authors for their reply, and for their comparison of their results with those for the G2019S variant in LRRK2 and its association with Parkinson's disease. As a well-established and well-studied pathogenic variant, LRRK2 G2019S is indeed an excellent example to illustrate two points:

1. Genuinely pathogenic variants are enriched in cases over controls. In 23andMe's cohort, the largest dataset for which we were able to find published allele counts, LRRK2 G2019S has an allele frequency of 0.09% in controls and 1.1% in cases, with an odds ratio of 9.6 (Do CB, 2011, see Table S1) for allele frequency breakdown by case/control status]. Another study of Europeans found an allele frequency of 0.8% in idiopathic PD cases (Gilks WP, 2005). The allele frequency in ExAC is accordingly 10-fold lower, 0.04% overall and 0.06% among non-Finnish Europeans. Among PD cases with a positive family history, the variant exhibits even stronger enrichment, with allele frequencies on the order of a few percent (Kachergus J, 2005, Nichols WC, 2005, Di Fonzo A, 2005). Results from other populations are in agreement: among North African Arabs and Ashkenazi Jews, the variant has an allele frequency of 0.6 - 1.4% in the general population, but a frequency of 9 - 19% in Parkinson's disease cases (Lesage S, 2006, Ozelius LJ, 2006).

2. Pathogenic variants may be found in ExAC, but at a frequency consistent with disease prevalence, penetrance, and allelic heterogeneity. Published estimates of lifetime risk of Parkinson's disease range from 3.7% (Elbaz A, 2002) to 6.7% (Driver JA, 2009). LRRK2 G2019S accounts for ~1% of all cases (as noted above), and is estimated to confer lifetime risk of ~32% (Ozelius LJ, 2006, Goldwurm S, 2007). From these figures, one can infer that allele frequencies even as high as 6.7% * 1% / 32% = 0.2% (a few times higher than what we see in ExAC) would not be surprising for this variant.

On both accounts, LRRK2 G2019S stands in contrast to NR1H3 R415Q. NR1H3 R415Q has no evidence of any enrichment in MS cases compared to population controls, and its allele frequency in ExAC is inconsistent with any but the lowest (<2%) penetrance when one considers that multiple sclerosis is less common than Parkinson's disease, and that this variant is found in only a small minority of cases. This contrast thus simply highlights the inadequacy of the genetic evidence supporting any role of NR1H3 R415Q in the etiology of MS.

Eric Minikel and Daniel MacArthur, Broad Institute of MIT and Harvard

On 2016 Jun 17, Carles Vilarino-Guell commented:

We read Mr. Minikel and Dr. MacArthur’s commentary with interest; although we respect their opinion, we are disappointed to see that it is solely based on two pieces of information. We would encourage anyone interested in the subject to read the whole article, assess all the scientific evidence with an open mind, and reach their own informed conclusion.

Regarding the two points of controversy, we would just like to highlight that in common complex neurological disorders it is not unusual for pathogenic mutations to have reduced penetrance, and to be present in the ExAC database. For example, in Parkinson’s disease (once considered to have no genetic component) widely variable penetrance estimates have been described (Trinh et al. 2014); and the LRRK2 G2019S mutation, which is irrefutably pathogenic, can be found in 47 individuals from the ExAC collection (rs34637584).

On 2016 Jun 13, Daniel MacArthur commented:

This paper reports that an NR1H3 variant, rs61731956, encoding p.Arg415Gln, causes familial multiple sclerosis (MS) (Wang Z, 2016). We have some major concerns about the evidence for the effect of the R415Q variant on risk for MS, which rests on two pedigrees with imperfect segregation with disease.

The reported data allows us to provide an estimate of penetrance for the R415Q variant. This variant was found in 1 out of 2053 individuals in a multiple sclerosis case series, but is also seen (as the authors note in passing) in 21 individuals among the 60,706 present in the Exome Aggregation Consortium (ExAC) collection (11-47290147-G-A). Enrichment in cases over controls is one important criterion for establishing pathogenicity of sequence variants (MacArthur DG, 2014, Richards S, 2015).

The ancestry distribution of the case series reported in Wang Z, 2016 is not stated, but the series was collected in Canada and appears to be of predominantly European ancestry (Sadovnick AD, 1998, Traboulsee AL, 2014). The 21 individuals with this variant in ExAC are all of non-Finnish European ancestry, with 66,738 non-Finnish European chromosomes having genotype calls for this variant. Thus, the variant is not significantly enriched in cases over ExAC population controls (P = .56) - indeed, its allele frequency is lower in MS cases (0.02%) than in ExAC European population controls (0.03%).

A review of lifetime risk estimates for MS found the best estimates of lifetime risk of MS to be 0.25% for women and 0.14% for men (Alonso A, 2008, see Table 1). Using the allele frequencies observed in ExAC and in the case series, along with these estimates of lifetime risk, we can apply formulae for calculating the penetrance or lifetime risk, and confidence intervals thereof, for reportedly Mendelian variants as described in (Kirov G, 2014, Minikel EV, 2016). The upper bound of the 95% confidence interval is 1.7% for women and 0.9% for men, indicating that this variant contributes extremely weakly, if at all, to MS risk.

The functional characterization of the effects of the p.Arg415Gln variant on gene function, while potentially interesting, does not provide independent support for a role of this gene in MS risk.

We urge the community to consider rigorous statistical approaches and independent replication before making strong claims of pathogenicity. In this case, publicly available data (and indeed data that are actually noted in the paper) are sufficient to strongly suggest that this variant has little or no effect on MS risk. Independent analyses of this variant in large case-control studies of MS are needed, and we look forward to seeing the results of such analyses from the MS community in the near future.

Eric Vallabh Minikel and Daniel MacArthur, Broad Institute of MIT and Harvard

On 2017 Sep 10, Misha Koksharov commented:

For rapidly changing metabolites (ATP, other nucleotides, etc), capturing their in vivo levels in brain tissues is particularly challenging (Heller HC, 2011, Wilson DF, 2011, Overmyer KA, 2015). They can change in a matter of seconds after the blood flow is interrupted. This unfortunately hampers investigation of many interesting questions (as well as leads to incorrect nucleotide measurements in quite a few papers). :( Hopefully, new techniques will become available and more widespread to overcome this.

On 2016 Dec 07, Joanne Kamens commented:

Thank you for this helpful set of definitions and clarifications. Common language will make the discussion more productive. I was prompted by an excellent blog by Hilda Bastian (http://blogs.plos.org/absolutely-maybe/2016/12/05/reproducibility-crisis-timeline-milestones-in-tackling-research-reliability/) and the subsequent twitter conversation to mention that these definitions don't address or even seem to mention the potential, influence or use of reagent/materials reproducibility. Experimental results and interpretation can be dramatically enhanced by the use of the correct standards, materials and/or reagents to reproduce a study. Protocol and methods sections alone are not sufficient to account for this as some reagents are not easily remade and are not always validated as being the same (unless subjected to quality control via repository storage or standard validation).

On 2016 Jun 15, thomas samaras commented:

This finding is consistent with other research that has found a positive correlation between height and coronary heart disease (Samaras, Mendall, Allebeck, Shapiro, Elsayed, Mori, Hameed and Gupta). Davenport studied about 1 million military recruits for WW I and found that taller recruits had more heart problems than shorter ones. WHO also reported that before 1900, coronary heart disease was rare in Europe and the US. Adults in 1900 were substantially shorter than they are today. Other studies have found taller people have higher atrial fibrillation, thromboembolism and pulmonary infarction. The Indian Heart Journal provides additional information on height and heart disease: Shorter height is related to lower cardiovascular disease risk--a narrative review.

On 2016 Aug 10, Joaquim Radua commented:

Re: the previous comments, please note that under the null hypothesis of no differences between groups, only 1 out of 20 studies should show differences between groups, which is absolutely not the case when randomizing coordinates or blocks of voxels. Random coordinates and similar approaches, which randomize the location of the findings rather than the individuals between groups, are not a valid way to exactly test this hypothesis. Rather, they are only used to yield approximated p-values that, appropriately thresholded, return a map similar but slightly more conservative than that of FWE-corrected p-values in mega-analyses. Voxel-based meta-analytic methods are young and there is room for improvement, but they are based on evidence.

On 2016 Aug 09, Christopher Tench commented:

This is not a result of confusion, but of the definition of statistical inference. Uncorrected p-values do not control the type 1 error rate. A meta-analysis is performed to improve estimates, and is a statistical problem demanding statistical methods. To threshold at an arbitrary p-value controls neither the FDR nor the FWE, so no quantitative evidence that the results are critical of the null hypothesis is available. You cant know if the results are true positives without doing the full experiment, but meta-analysis is used for the case where the full experiment (mega analysis) has not been done. The one, and only, thing that can be done is to make sure that the null hypothesis is appropriately rejected; arguably the whole point of statistical inference. That requires either FWE or FDR control. Using just random coordinates and an uncorrected p-value will produce results that are apparently publishable, but obviously incorrect. Without any estimate of error rate, there is no quantifiable evidence that the results are meaningful.

On 2016 Aug 01, Joaquim Radua commented:

Re: the previous comment, I think there may be some unfortunate confusion. Raw p-values of current voxelwise meta-analyses have not the same meaning as usual p-values because they are not derived from the usual null hypothesis (“there are no differences between groups”), but from another null hypothesis (“all voxels show the same difference between groups”). Thus, up to the moment one of the only ways to "approximately" know if the results of a voxelwise meta-analysis are neither too liberal nor too conservative is to compare them with the results of a mega-analysis of the same data, and that's what it was done.

On 2016 Jun 13, Christopher Tench commented:

The methods employed in this study are not corrected for multiple statistical tests; uncorrected p-value. This can therefore not be considered a meta-analysis as there is no evidence of statistical significance. The results can not be considered valid.

On 2016 Jun 13, Christopher Tench commented:

The version of GingerALE used (2.3.4) had a bug that produced false positive results. This was fixed at version 2.3.6

On 2016 Oct 29, David Keller commented:

Why MACRA seems unlikely to reduce costs or improve the quality of medical care

The Centers for Medicare and Medicaid Services (CMS) recently released nearly 2400 pages of new regulations, comprising the "Medicare Access and CHIP Reauthorization Act" (MACRA). The acronym "MACRA" actually contains another acronym within it (an homage to Kafka?), "CHIP", for Children's Health Insurance Program. New regulations introduced by MACRA mandate fundamental changes in the way doctors are paid and how they provide medical services. The following is but a partial description:

1) The main purpose of MACRA is to reduce the cost of Medicare, after the failure of the Sustainable Growth Rate (SGR) law to do so. However, the demand for real medical services, such as hernia repairs, cataract extractions, colonoscopies, skin biopsies, chemotherapy, etc., will continue to grow, while MACRA will funnel out money to clinicians for endless reports on quality metrics, and to entrepreneurs, for participating in risk-sharing ventures. Diversion of scarce CMS funds away from direct clinical care services can only worsen the looming Medicare financial collapse, and the entrepreneurs are likely to get the best of the risk-sharing ventures.

2) CMS states that physician quality-of-care bonuses will be financed with fines obtained from other doctors, however this "zero sum" financial balance is only achieved in a static model. The losers in this system, who will sustain fines that substantially reduce their Medicare reimbursements, will have their already-thin operating margins cut further, and many may decide to stop accepting Medicare. As they drop out, physicians farther up the "quality of service" food chain will see their quality score percentile ranks diminish in turn, creating a vicious cycle of [increasing fines] >>> [reduced net payments] >>> [physician dropouts] >>> [increasing fines]. Thus, in the dynamic model, fines cannot finance bonuses because doctors who pay fines will be more likely to drop out of the system, eliminating themselves as a source of funds to pay for bonuses.

3) MACRA seems designed to nurture a cohort of doctors who excel at tasks like reporting patient satisfaction surveys, and reducing costs by not performing clinical services. The demand for clinical services will not go away, however, and if paying for required medical services is causing Medicare losses, these losses can only worsen when the cost of running the MACRA bureaucracy is piled-on.

4) The original fee-for-service Medicare system is preferred by both doctors and patients, partly due to the choices and autonomy it allows for both patients and doctors. Fee-for-service also aligns the interests of the patient, who wants to receive the best medical care appropriate for his condition, with those of the physician, who wants to earn a living by providing medical care. The problem, of course, is costly overtreatment, but this is well-policed by CMS, and by the natural reluctance of patients to undergo procedures. When working for capitated fees, physicians are incentivized to provide less care, which can result in undertreatment, a far more dangerous and insidious situation, and harder to police. How will CMS guard against physicians becoming too parsimonious with their capitation money, and doing too little for their patients?

5) MACRA was implemented nationwide too quickly, rather than studied, debated and rolled out slowly. Physicians were given 2 months to study the final MACRA document during the public comment period, requiring them to read 40 pages of dense regulatory language per day, every day, during that period. Such massive changes require sufficient time to fully educate doctors about their effects. We await evidence that these measures actually reduce costs or increase quality of care.

In summary, MACRA will increase spending on non-clinical activities, worsening the financial status of Medicare. MACRA is overly intrusive in dictating how doctors should practice medicine, basing physician incentives on their performance of incentivized services, which may not be relevant or beneficial in some individuals or subpopulations. MACRA introduces capitation to original Medicare, a payment system which disincentivizes clinical services by making them an expense to the physician, rather than a source of income. Based on the above considerations, MACRA seems unlikely to improve either the financial prospects or the clinical quality of Medicare.

On 2016 Jul 24, James Yeh commented:

Editor's Comment

E-Cigarettes and Smoking Cessation — Polling Results

James S. Yeh, M.D., M.P.H., and Edward W. Campion, M.D.

The goal of smoking cessation is to reduce the effect of smoking on mortality and morbidity. Despite the dramatic reduction in the prevalence of smoking over the past 50 years, the use of tobacco still contributes significantly to morbidity and mortality.[1] About 1 in 5 deaths in the United States each year can be attributed directly or indirectly to cigarette smoking, and life expectancy among smokers is 10 years shorter than that among nonsmokers.[2] Quitting before 40 years of age reduces the risk of dying from smoking-related illness by 90%.[3]

The use of e-cigarettes has become more prevalent; 16% of U.S. high-school students use e-cigarettes,[4] and about 4% of U.S. adults use e-cigarettes on a regular basis.[5] There is no doubt that e-cigarettes engender tremendous interest from the public, the medical community, and the Food and Drug Administration (FDA) with respect to their potential as lifesaving nicotine-delivery devices for persons with tobacco dependence. Recent modeling research has projected a 21% reduction in death attributed to tobacco-smoking if e-cigarettes are used.[6] However, despite the potential benefit, there remain concerns about the safety of e-cigarettes and their efficacy for smoking cessation; for this reason, the FDA has recently extended its regulatory authority to cover e-cigarettes.[7]

In June, we presented the case of Mr. O’Malley, a 29-year-old man who had been smoking since he was 15 years of age and more recently was smoking up to 1.5 packs per day.[8] Readers were invited to vote on whether to recommend that Mr. O’Malley try using e-cigarettes for smoking cessation. Mr. O’Malley had a history of obesity, hypertension, and childhood seizures. In the past, he had quit smoking “cold turkey” and had used various nicotine-replacement therapies, but he had never been able to sustain smoking abstinence for an extended period of time.

More than 35,000 readers viewed the Clinical Decision case, and 666 readers in 62 countries responded to the poll. The largest group of respondents, representing more than 45% of the votes, was from the United States and Canada (306 voters), followed by respondents from Europe (235). Two thirds of the respondents (66%) voted to recommend the use of e-cigarettes for smoking cessation, and the remaining respondents voted against recommending e-cigarettes. This result suggests that a majority of the poll respondents believe that e-cigarettes are a reasonable strategy for smoking cessation, at least for a patient such as the one described in the case vignette.

A substantial proportion of the 41 voters who submitted comments emphasized the health benefits derived from reducing or quitting tobacco use. These benefits, which include a lowering of the risks of cardiovascular-related death, lung cancer, and pulmonary symptoms, are especially important in this case of an asymptomatic young tobacco-dependent smoker who already has some risk factors for cardiovascular disease.

Commenters raised several related recurring themes. They emphasized the difficulty of current smoking-cessation regimens, such as nicotine-replacement therapy, varenicline, or bupropion, in sustaining smoking abstinence. A number of commenters considered e-cigarettes to be a reasonable complementary smoking-cessation aid to kick-start the process of reducing and quitting tobacco smoking, along with the more traditional smoking-cessation aids, with the eventual goal of weaning smokers off e-cigarette use altogether.

Many readers also commented that the quality and the safety of the e-cigarettes could not be relied on, since e-cigarettes were an unregulated nicotine-delivery device at the time. Some readers indicated their belief that e-cigarettes should be regulated by the FDA and that e-cigarettes should not be recommended until there is reliable evidence that they are less hazardous than tobacco smoking and that they are efficacious in reducing tobacco smoking. Others expressed concern about the availability of e-cigarettes to adolescents. Thus, for some, uncertainty about the performance and safety of e-cigarettes deterred them from recommending their use as a smoking-cessation aid.

REFERENCES

[1] Fiore MC, Baker TB. Treating smokers in the health care setting. N Engl J Med 2011;365:1222-31. [2] 2014 Surgeon General’s report: the health consequences of smoking—50 years of progress. Atlanta: Centers for Disease Control and Prevention (http://www.cdc.gov/tobacco/data_statistics/sgr/50th-anniversary/index.htm). [3] Jha P, Ramasundarahettige C, Landsman V, et al. 21st-Century hazards of smoking and benefits of cessation in the United States. N Engl J Med 2013;368:341-50 [4] Singh T, Arrazola RA, Corey CG, et al. Tobacco use among middle and high school students — United States, 2011 ̶ 2015. MMWR Morb Mortal Wkly Rep 2016;65:361-7. [5] Schoenborn CA, Gindi RM. Electronic cigarette use among adults: United States, 2014. NCHS data brief no. 217. Hyattsville, MD: National Center for Health Statistics, 2015. [6] Levy DT, Borland R, Villanti AC, et al. The application of a decision-theoretic model to estimate the public health impact of vaporized nicotine product initiation in the United States. Nicotine Tob Res 2016 July 14 (Epub ahead of print). [7] FDA takes significant steps to protect Americans from dangers of tobacco through new regulation. Silver Spring, MD: Food and Drug Administration, May 5, 2016 (http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm499234.htm). [8] Yeh JS, Bullen C, Glantz SA. E-cigarettes and smoking cessation. N Engl J Med 2016;374:2172-4.

On 2016 Jun 30, Riccardo Polosa commented:

In this case study of a 29-yr old man (Mr. O'Malley) interested in giving up smoking using e-cigarettes, two experts are providing their personal recommendations.

Expert no.1, Dr Christopher Bullen, is recommending e-cigarettes for smoking cessation because he thinks that Mr. O’Malley has limited pharmacologic treatment options (contraindications for bupropion and varenecline prescription due to his history of seizures; inability to abstain from smoking on several occasions despite being on NRT).

Expert no.2, Dr Stanton Glantz, is not recommending e-cigarettes because the results of his - flawed (1) - metaanalysis shows that these products are not proven to assist smoking cessation (2).

The case study presented here is more challenging than actually appears and that for several reasons.

Mr. O'Malley is relatively young and quit rates in young adults are know to be very low as proven by the Mr. O’Malley’s history of frequent relapses. As there is no evidence demonstrating efficacy of FDA-approved smoking cessation drugs for young adults, these cannot be recommended in this specific age group.

Mr. O’Malley’s is at high risk of relapse and very little can be done to manage smokers with a history of frequent relapses (3).

Mr. O’Malley is overweight and has hypertension. Stopping smoking is known to lead to weight gain (4). Hence, it is important to consider that Mr. O’Malley - if successful - will have to deal with the burden of post-cessation weight gain with its important negative health consequences, particularly in consideration of the fact that obesity and hypertension are well known risk factors for cardiovascular disease.

Mr. O’Malley would benefit from switching to a much cleaner source of nicotine. His personal preference for the e-cigarettes should be respected and the health care provider should offer a balanced overview of their risk/benefit ratio (5,6).

More specifically, relevant to Mr. O’Malley’s case, it worth noticing that: 1) e-cigarettes have helped abstaining from conventional cigarette young adults as well (7); 2) e-cigarettes have been shown to reduce post-cessation weight gain in quitters (8); 3) smokers with elevated blood pressure who quit by switching to e-cigarettes may lower their BP in the long-term (9).

Ref.

1. Hajek P, McRobbie H, Bullen C. E-cigarettes and smoking cessation. Lancet Respir Med. 2016 Jun;4(6):e23. doi: 10.1016/S2213-2600(16)30024-8. Epub 2016 Apr PubMed PMID: 27133216.
2. Kalkhoran S, Glantz SA. E-cigarettes and smoking cessation in real-world and clinical settings: a systematic review and me- ta-analysis. Lancet Respir Med 2016;4:116.
3. Caponnetto P, Keller E, Bruno CM, Polosa R. Handling relapse in smoking cessation: strategies and recommendations. Intern Emerg Med. 2013 Feb;8(1):7-12.
4. US Department of Health and Human Services JL. Health People . Washington, DC: US Government Printing Office; 1990.
5. Farsalinos KE, Polosa R. Safety evaluation and risk assessment of electronic cigarettes as tobacco cigarette substitutes: a systematic review. Ther Adv Drug Saf. 2014 Apr;5(2):67-86.
6. Public Health England. E-cigarettes: An evidence update. London: Public Health England, 2015.
7. Choi K, Forster J. Characteristics associated with awareness, perceptions, and use of electronic nicotine delivery systems among young U.S. Midwestern adults. Am J Public Health. 2013;103(3):556–561.
8. Russo C, Cibella F, Caponnetto P, Campagna D, Maglia M, Frazzetto E, Mondati E, Caruso M, Polosa R. Evaluation of Post Cessation Weight Gain in a 1-Year Randomized Smoking Cessation Trial of Electronic Cigarettes. Sci Rep. 2016 Jan 5;6:18763. doi: 10.1038/srep18763
9. Farsalinos K, Cibella F, Caponnetto P, Campagna D, Morjaria JB, Battaglia E, Caruso M, Russo C, Polosa R. Effect of continuous smoking reduction and abstinence on blood pressure and heart rate in smokers switching to electronic cigarettes. Intern Emerg Med. 2016 Feb;11(1):85-94.

On 2016 Jun 16, David Keller commented:

Rarely-laundered lab coats and neckties risk transmission of harmful bacteria

It is all well and good that patients prefer their doctors to wear white lab coats, but the safety and well-being of patients demands they not be exposed to the highly unsanitary white coat which is laundered only once a week or even once a fortnight. And when, if ever, was the last time you autoclaved your neckties? Never, like me. And think of all the bacteria the distal tip of your tie has picked up over the years. No, my colleagues, our attire must only consist of freshly-laundered garb each day: if you wore it before, then wear it no more. The necktie must be forever banished from clinical areas. Bow-ties are creepy and clearly a form of nerd micro-aggression; therefore, we must wear our clean shirts open at the neck, regardless of how casual and relaxed this may cause us to feel, or to be perceived.

On 2016 Jun 04, H Horvath commented:

Many questions arise.

Did you obtain human subjects research approval for this study (and its interview questions and questionnaires) from your institutions? Did the school district's Board give approval? How did you come up with your interview questions and questionnaire? How did you validate them? It would also have been useful to see the consent form these parents signed -- I wonder whether they really understood that you would be introducing such disorienting, disturbing concepts to young children. Two year study -- Over how many cumulative hours, days, weeks were the interviews and "guided group questionnaire" sessions conducted?

You present only year 2 results, simply saying that "outcome measures were not included in the 1st year." Do you mean to say that the first year was a complete waste of everyone's time and that you have no data to report? Or do you mean that you are planning to report year one results for different outcomes in a separate paper? As formulated, your explanation for the absence of year one outcome data in the current paper suggests selective outcome reporting.

What possible good could come from making small children endure these confusing interviews and "group questionnaire" sessions with "trained researchers"? Who were these "trained researchers"? What proportion of parents didn't consent for their children to participate? What were characteristics of consenting vs. non-consenting parents? What proportion of consenting parents did not further respond? What were parents told of their children's specific responses? What happened with kids whose parents didn't consent? Were they included in the study anyway, filling out questionnaires in groups and being interviewed? You do not describe other activities given to non-consented children, and there is no indication that these activities took place outside ordinary school hours -- which suggests that non-consented children were brought along for the ride. What were children's thoughts and concerns about the interviews and questionnaire sessions? What clarifying questions did they ask of the "trained researchers"? What measures did you take to protect child privacy? Did children benefit from this study, and if so, how? Did you follow up to learn of any harmful effects on children? What were these effects?

What does any of this have to do with reading, writing, arithmetic, geography, history etc.? Never mind biology, in which kids still might learn that in humans, there is a male sex and a female sex, with very few exceptions (i.e. in the case of disorders of sexual development). "Gender" is merely sex role stereotyping. It is not good to instill in young children the false notion that performing stereotypes of appearance, behavior and mannerisms normally used by the opposite sex will somehow enable them to change sex. This wasn't the "goal" of your study, I know, but the children no doubt are now a lot more confused about themselves. You can't conduct "gender" programming like this in small children and expect that the world will applaud it, despite ethical sloppiness or actual lapses. Children are not mere guinea pigs for social programming research. To be fair to your paper, most other "gender" studies in children are ethically-challenged, to say the least.

On 2016 Jun 03, Ewen Gallagher commented:

An Extra View article related to our published analysis investigating the role of Mekk1 (encoded by Map3k1) in T cells using Map3k1^ΔKD and Lck^Cre/+ Map3k1^f/f mice (1). Examines the role of Mekk1 in Th17 differentiation, and includes a comparison between the roles of Mekk1 and Tak1 (encoded by Map3k7) in the liver T cells using Map3k1^ΔKD, Lck^Cre/+ Map3k1^f/f and Lck^Cre/+ Map3k7^f/f mice.

Related research. (1). Suddason T, Anwar S, Charlaftis N, Gallagher E. T-Cell-Specific Deletion of Map3k1 Reveals the Critical Role for Mekk1 and Jnks in Cdkn1b-Dependent Proliferative Expansion. Cell Rep 2016; 14:449-57.

On 2016 Jun 03, Clive Bates commented:

There are several weaknesses in the reasoning in this paper.

1. Singh et al assert: “exposure to e-cigarette advertisements might contribute to increased use of e-cigarettes among youths.” The study is cross-sectional and does not (and cannot) establish that advertising causes e-cigarette use. Even if the authors acknowledge that their study does not establish a causal relationship, that has not stopped them drawing conclusions as if it does.

2. The finding that higher advertising exposure would be associated with greater e-cigarette use is quite likely, but there are many possible explanations. It could be that e-cigarette users see more advertising because of the way they live and their interests direct them to where such advertising is visible. Alternatively, teenagers already using the products may just be more interested and have better recall of advertising they have seen, even if exposure is no different.

3. Marijuana is not advertised but its prevalence among high school students is higher (23.4%) than tobacco (22.4%) on the basis generally used by CDC (used at least once in the last 30 days) - see CDC Youth Risk Behavior Surveillance — United States, 2013. It may be that better explanatory variables than advertising exposure are needed to interrogate prevalence of youth risk behaviours.

4. Even if we allow that e-cigarette advertising may be increasing youth e-cigarette uptake, something that is far from established, the authors need to consider the likelihood that this is displacing cigarette smoking. Given that independent risk factors for teenage vaping and smoking are likely to be similar, it would be surprising if vaping was not adopted as an alternative to smoking, at least in some cases. In this event, the health effect of e-cigarette use is positive, and, therefore, so is any role that advertising plays in it.

5. The observed sharp decline in teenage cigarette smoking that has coincided with the sharp rise in teenage e-cigarette use does not establish a causal relationship between these trends, but it does suggest that it is a plausible hypothesis that vaping is reducing smoking and therefore that great care should be taken when designing policies that may attenuate this effect. See data: CDC Tobacco Use Among Middle and High School Students — United States, 2011–2014.

6. The authors jump to a policy conclusion that goes far beyond the limitations of their study: "Multiple approaches are warranted to reduce youth e-cigarette use and exposure to e-cigarette advertisements”.

7. Before making such policy proposals, the authors should consider several issues and potential unintended consequences beyond the scope of this paper: that young people might smoke instead of using e-cigarettes; that e-cigarette advertising might be an effective form of anti-smoking advertising; that such restrictions may reduce adult switching from smoking to vaping and thus create damage to adult health; that excessive restrictions on advertising protect incumbent industries (cigarettes) and reduce the returns to innovation in the disruptive entrants (e-cigarettes).

8. The assertive and unqualified nature of the policy proposal raises concerns of an unacknowledged investigator bias. This would not be that surprising. The authors are from CDC, and the media release that accompanied this survey abandoned all caution about causality or unintended consequences. It reflected the CDC’s adversarial prior policy stance on these issues, none of which are supported by the survey itself:

“The same advertising tactics the tobacco industry used years ago to get kids addicted to nicotine are now being used to entice a new generation of young people to use e-cigarettes,” said CDC Director Tom Frieden, M.D., M.P.H. “I hope all can agree that kids should not use e-cigarettes.”

Conclusions: CDC should adopt a more dispassionate and careful approach to both reporting and communicating survey findings. In publishing for peer-reviewed journals, CDC staff should disclose relevant CDC policy and advocacy positions as non-financial competing interests.

On 2017 Mar 06, Misha Koksharov commented:

A really nice FP tool and screening/engineering work.

I have a few questions:

1) "Bioluminescence characterization in vitro. Each protein was diluted to 2 nM in 30mM MOPS 100mM KCl pH 7.5 with 0.1% BSA, and a final concentration of 20 μM substrate was used."

a) Do you mean you have used 20 μM for both LH2 and ATP in case of firefly luciferase (Ppy luc2)?

b) Was Mg²⁺ added to the reaction buffer in case of Fluc?

2) "Hexahistidine-tagged proteins were expressed in E. coli at 30°C for 30 h"

How active was Fluc after growing E.coli at 30°C? I'd expect that this should noticeably decrease its specific activity compared with growing at 18-22°C. (Although, this is usually not important in mammalian cells where chaperones compensate for thermolability of Ppy luc2).

3) Comparison of Nanoluc, Antares vs Fluc in live cells, tissues and animals.

What would be the difference when corrected for protein and mRNA half-life - essentially, for actual protein levels in cells during recording/imaging? For example, Fluc luc2 has a protein half-life of 5-8 hours while NanoLuc is much more stable.

4) Do you know what are the actual concentrations of Nanoluc, Fluc, etc in live mammalian cells in relevant experiments (in μM, for example)? So far it seems like no one knows, even Keith Wood himself. :(

On 2016 Jul 25, Yonwoo Jung commented:

Thanks very much for your comment. We just found there is wrong description about sample size. Actual sample size is that 11 offsprings from 3 saccharin receive dams and 14 offsprings from 3 nicotine received dams. We will contact to publisher and try to correct this one. I really appreciate your comment. I also checked your paper about psedoreplication. It is excellent point of view. I will keep that in mind for future experiment. We are very welcome for any comment. Thanks again for your comment.

On 2016 Jul 02, Stanley Lazic commented:

The results in this paper are hard to interpret for two reasons. First, the sample size has been incorrectly determined in most experiments, and thus the p-values are artificially too small. This is the problem of pseudoreplication (Lazic SE, 2010, Vaux DL, 2012). For example, the experiment in Fig 1b uses 25 slices from 10 brains (animals), and the appropriate sample size is 10, not 25.

Second, in some experiments (e.g. Fig 1b again) nicotine was applied to pregnant dams and the effects were measured in the offspring. Here, the sample size is the number of dams and not the number of offspring (Lazic SE, 2013, Lazic SE, 2013, and references therein). The number of dams was not mentioned, and so the worst case scenario is that the 5 control mice are from one litter and the 5 nicotine mice are from another, making the sample size 2.

Nature Neuroscience should be commended for their detailed reporting requirements, which allow readers to better evaluate the evidence.

On 2017 May 24, Jordan Anaya commented:

This paper has been retracted: http://retractionwatch.com/2017/05/24/authors-retract-much-debated-blockchain-paper-f1000/

On 2017 Mar 09, Daniel Himmelstein commented:

Irreproducible Timestamps

Please see my blog post describing why Irving & Holden's timestamping implementation is broken. According to Carlisle's method, there is no record in the Bitcoin blockchain proving the existence of the clinical trial protocol from this study.

WARNING: Users should not rely on Irving & Holden's method for producing blockchain-verifiable timestamps. Doing so could expose them to invalid timestamps, accusations of fraud or misconduct, financial loss, and sabotage. As discussed in my blog post, there are several established solutions for Blockchain timestamping that have been reviewed by expert cryptographers. I strongly encourage users to rely on one of these methods and to disregard this study from Irving & Holden.

Feel free to reference my blog post:

Daniel S. Himmelstein (2017) "The most interesting case of scientific irreproducibility?" Satoshi Village. http://blog.dhimmel.com/irreproducible-timestamps/

Or the GitHub analysis behind it:

Daniel Himmelstein (2017) "dhimmel/irreproducible-timestamps v1.0: Initial replication analysis for the Satoshi Village blog post" Zenodo. DOI: 10.5281/zenodo.375952 https://doi.org/b2tz

I will update this comment if Irving & Holden provide cryptographic proof of their timestamp and the validity of their method. Until then, I strongly urge readers to avoid advertising or building on this study.

On 2016 Jul 20, Jordan Anaya commented:

Readers of this article may be interested in this post at Retraction Watch or this post by Neuroskeptic.

On 2016 Jun 15, thomas samaras commented:

Many studies indicate that short, lean people have much lower risk of coronary heart disease compared to taller people. Scores of studies find either no relation between height and CHD or a positive relation between height and CHD. Studies finding taller height is related to increased CHD include the works of Shapiro, Mendall, Mori, Elsayed, Allebeck, Hameed and Gupta. Other studies have found taller people have higher rates of abdominal aortic aneurysm, arial fibrillation, thromboembolism and pulmonary infarction. The only populations that have been found to be free of CHD or stroke are Solomon islanders, Papua New Guinea, Kalahari bushmen, Congo pygmies, and Kitavans. These populations range from less than 5 feet to about 5 feet 6 inches. (These findings are based on 20th C studies.) A report by the World Cancer Research Fund found that the process of industrialization has increased our height, weight and chronic diseases (including CHD). This report also stated that the Western diet has promoted chronic diseases. Trowell and Burkitt also reported that pre-western populations following their traditional diets were free of major chronic diseases common in the West. The studies finding shorter people have more CHD are confounded by a number of factors, including overweight, smoking, diet,lack of exercise, socioeconomic status, and catch up growth when in early childhood due to low birthweight. Rapid growth in childhood has been found to be related to chronic diseases at older ages. The Japanese experience shows the weakness of studies showing shorter people have higher CHD; for example, Mainland Japanese are shorter than Hawaiian Japanese and have substantially higher CHD. In addition, Hawaiian Japanese are shorter than Californian Japanese and have lower CHD.

On 2016 May 31, David Keller commented:

None

On 2016 May 30, David Keller commented:

My ColoGuard cancer screening was reported as "negative" in April, 2016. I immediately requested the results of two tests done as part of the ColoGuard screening algorithm, specifically my fecal hemoglobin concentration, and my ColoGuard Composite Score [of colon cancer risk]. These are important tests with easily interpreted results, which every patient's physician should review.

Exact Sciences, the manufacturer and retail seller of Cologuard screening tests, stated that the test results I requested cannot be reported due to FDA regulations, which require stand-alone approval of all tests, even those used as part of an approved screening kit. The test results I requested, although conveying unique, accurate and medically important information, thus cannot be reported to the patient or his doctor, even if severely abnormal results are found.

Patients have an ethical right to demand all of their test results, even for tests which are performed without specific FDA approval, but as part of an FDA-approved algorithm of tests.

The regulations being enforced at this time will lead to avoidable patient harms, and must be revised to mandate patient access to the results of all medical tests performed on them.

On 2016 May 29, David Keller commented:

Please see the final version of my article, which is published online above. I posted it here temporarily during a dispute with an affected commercial entity.

On 2016 May 29, David Keller commented:

Additional comments and suggestions related to MultiTarget fecal testing"

The MultiTarget composite score can be negative for colon neoplasia despite an elevated fecal hemoglobin concentration consistent with another disorder.[1] In such cases, FDA regulations prohibit release of any MultiTarget test data except whether the overall screening test was positive or negative for colon neoplasia.[2]

I screened myself with the MultiTarget screening method because it is more sensitive and has a smaller false negative rate than FIT screening, leading to fewer missed cancers. I intend to screen myself again with MultiTarget the next time I am eligible for full Medicare reimbursement in 3 years. The down side is that MultiTarget has a lower specificity and a higher false positive rate than FIT, resulting in more unnecessary colonoscopies with MultiTarget; this is a trade-off I chose to accept. Because Medicare (with a standard Medigap policy) pays for MultiTarget screening in full, its higher cost than FIT was not an issue for me.

Additionally, I will screen annually with a FIT test, to detect occult fecal hemoglobin elevations not reported by MultiTarget. Performing FIT testing in parallel with MultiTarget testing in this way will result in even higher sensitivity and lower specificity, but less so than predicted by the "parallel testing" formula, since these two tests are not completely independent.

I have written to the FDA requesting them to allow the makers of MultiTarget to release all the intermediary data generated by their test, including the patient's composite score, fecal hemoglobin and all DNA test results. It is not ethical or in the public interest for the FDA to refuse to allow patients or their physicians to have access to this data, in my opinion.

References

1: Keller DL, Patients Should Not Be Denied Access to Their Test Results, online publication, temporarily removed, American Journal of Medicine, May 2016

2: Telephone discussion with MultiTarget laboratory director, May 2016

On 2017 Apr 13, Randi Pechacek commented:

Holly Ganz wrote a blog post about this paper and the significance of its findings on MicroBEnet. Read about it here

On 2016 Jul 04, Stephen Tucker commented:

This version of the article contains a production error: labels and annotations are missing in Figure 1A-B and Figure 5A. This has been pointed out to the publishers and will be corrected soon.

On 2016 May 30, Simon McGrath commented:

Authors’ abstract

(An abstract wasn’t required for this editorial, but the authors felt it would be helpful to add this one for PubMed. I am one of the authors.)

Myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) is as prevalent and disabling as multiple sclerosis, diabetes and rheumatoid arthritis but biomedical research into the condition has been grossly underfunded. However, the NIH is committed to ramping up funding for this illness and there are now promising research leads. We indicate clues to the biological mechanisms that may perpetuate the condition and suggest the key elements of a concerted research programme. In the absence of effective treatments, study of mechanism remains a key priority including the possible role of stochastic factors.

We identify the three most interesting major categories of causal model: (1) the brain is responding normally and symptoms are due to persistent signal input from peripheral tissues, such as cytokines or metabolites; (2) there is a persistent abnormality of ‘housekeeping’ processes in the brain, such as an increase in activation of microglia; and (3) there is a persistent abnormality in neural signalling in sensory pathways.

We recommend seven important, practical steps to make progress towards effective treatments: (1) build research infrastructure, including population-based cohorts with close attention to diagnostic criteria; (2) use new, developing techniques for brain imaging; (3) pursue promising immunological leads, such as natural killer cell abnormalities, cytokine shifts, auto-antibodies, and immune responses to viruses such as Epstein-Barr virus; (4) further explore autonomic/endocrine regulation; (5) extend the research on post-exertional changes in physiology; (6) attempt to replicate key findings to determine which are robust; and (7) present data in a way that allows subgroup analysis, and always provide raw data.

We conclude that the problem of ME/CFS now looks solvable and we call on the wider biomedical research community to target the condition.

On 2016 May 29, Claudiu Bandea commented:

Remarkable results, questionable report

“Our findings raise the intriguing possibility that β-amyloid may play a protective role in innate immunity and infectious or sterile inflammatory stimuli may drive amyloidosis” (1). Indeed, fascinating findings. What Kumar et al. did not articulate, though, is that their result is one of many findings, observations, and arguments supporting the theory (2,3) that:

(i) β-amyloid, tau, α-synuclein, huntingtin, TDP-43, prion protein and other primary proteins implicated in neurodegenerative diseases are members of the innate immune system;

(ii) The isomeric conformational changes of these proteins and their assembly into various oligomers, plaques, and tangles are not protein misfolding events as defined for decades, nor are they prion-replication activities, but part of their normal, evolutionarily selected innate immune repertoire;

(iii) The immune reactions and activities associated with the function of these proteins in innate immunity lead to Alzheimer’s, Parkinson’s, Huntington’s, ALS and Creutzfeldt-Jakob Disease, which are innate immunity disorders.

Generating data and observations, although essential, represents only half of the scientific process; the other is their interpretation and integration into the existing knowledge and paradigms. That’s where the article by Kumar et al. falls short.

Perhaps the authors were not fully familiar with the literature and paradigms in the field of neurodegenerative diseases. Or, perhaps, Kumar et al. did not consider it relevant to discuss their results in the context of previous findings, ideas and hypotheses. For example, the authors did not address or explain their results in context of the ‘prion’ paradigm, which has dominated the thinking in the field of Alzheimer’s and other neurodegenerative diseases in the last few years (e.g. 4-7). Nor did they refer to a related study entitled “Alpha-synuclein expression restricts RNA viral infections in the brain” (8), which is highly relevant considering the fact that alpha-synuclein, a putative member of the innate immune system and the primary protein implicated in Parkinson’s, is a significant player in Alzheimer’s disease. Also, some might consider highly questionable leaving out the study by Kobayashi et al. entitled “Binding sites on tau proteins as components for antimicrobial peptides” (9).

Given these omissions, it's no wonder in her The New York Times article on Kumar et al. study, Gina Kolata wrote: “The Harvard researchers report a scenario seemingly out of science fiction”.

References:

(1) Kumar et al. 2016. Amyloid-β peptide protects against microbial infection in mouse and worm models of Alzheimer's disease. Sci Transl Med. 25;8(340); Kumar DK, 2016

(2) Bandea CI. 2013. Aβ, tau, α-synuclein, huntingtin, TDP-43, PrP and AA are members of the innate immune system: a unifying hypothesis on the etiology of AD, PD, HD, ALS, CJD and RSA as innate immunity disorders. bioRxiv. doi: 10.1101/000604; http://biorxiv.org/content/biorxiv/early/2013/11/18/000604.full.pdf

(3) Bandea CI. 2009. Endogenous viral etiology of prion diseases. Nature Precedings; http://precedings.nature.com/documents/3887/version/1/files/npre20093887-1.pdf

(4) Frost B, Diamond MI. 2010. Prion-like mechanisms in neurodegenerative diseases. Prion. 1(3):155-9; Frost B, 2010

(5) Nussbaum JM, Seward ME, Bloom GS. 2013. Alzheimer disease: a tale of two prions. Prion. 7(1):14-9; Nussbaum JM, 2013

(6) Watts JC et al. 2014. Serial propagation of distinct strains of Aβ prions from Alzheimer's disease patients. Proc Natl Acad Sci U S A. 11(28):10323-8; Watts JC, 2014

(7) Jaunmuktane et al. 2015. Evidence for human transmission of amyloid-β pathology and cerebral amyloid angiopathy. Nature. 525:247-50; Jaunmuktane Z, 2015

(8) Beatman et al. 2015. Alpha-Synuclein Expression Restricts RNA Viral Infections in the Brain. J Virol. 90(6):2767-82; Beatman EL, 2015

(9) Kobayashi et al. 2008. Binding sites on tau proteins as components for antimicrobial peptides. Biocontrol Sci. 13(2):49-56. Kobayashi N, 2008

(10) Kolata G. 2016. Could Alzheimer’s Stem From Infections? It Makes Sense, Experts Say. The New York Times; May 25, 2016.http://www.nytimes.com/2016/05/26/health/alzheimers-disease-infection.html?_r=0

On 2016 Oct 18, Konstantinos Farsalinos commented:

I am surprised that a completely irrelevant to the study (or to my post) comment is published here. I am satisfied to see that Hong Yin had no comment whatsoever on the content of my analysis on the problems of the paper by Ji et al. (1). I invite Hong Yin to comment on anything inappropriate or inaccurate, and of course i will respond accordingly and even correct any inaccuracies in my analysis. Instead of doing that, Hong Yin characterized the critical approach to studies with major methodological problems, which is a fundamental purpose of science and the duty of a scientist, as an "attack". There is no need to further comment on that, especially since i am in the process of finalizing the comprehensive and detailed replication of the New England Journal of Medicine research letter, which will provide further justification for the well constructed analysis on the need for retraction (2).

The disclosure of any COI was not only transparent from my side but also included declaration of funding of the institute which took place more than 3 years ago. According to ICMJE guidelines, it was not necessary to declare this, but i did it for the shake of full transparency. Thus, the initial sentence in the comment by Hong Yin is unjustifiable.

Additionally, Hong Yin omitted to mention that a very detailed response to the alleged "problems" of my paper has been published in response to the Shihadeh et al. commentary, both by the authors of the original study (3) and by the editor of Addiction journal (4). In my response, i clearly presented the case that our study (5) evaluated the widely-known organoleptic parameter of dry puff detection in e-cigarettes. This phenomenon can easily result (and unfortunately is frequently resulting) in abuse of e-cigarettes in the laboratory and misleading reports on e-cigarette emissions which are irrelevant to human exposure. The authors of the letter characterized dry puffs as a term with "tenuous ontological status", despite the fact that this phenomenon has been described in the literature since 2013 (6) and is well-known and explained by e-cigarette users long before that.

The act of writing a comment is irrelevant when the content of the criticism is not examined. I will be happy to accept and respond to any criticism about the content of my research or my comments.

References:

1. Ji EH, Sun B, Zhao T, Shu S, Chang CH, Messadi D, Xia T, Zhu Y, Hu S. Characterization of Electronic Cigarette Aerosol and Its Induction of Oxidative Stress Response in Oral Keratinocytes. PLoS One. 2016 May 25;11(5):e0154447.

2. Bates CD, Farsalinos KE. Research letter on e-cigarette cancer risk was so misleading it should be retracted. Addiction. 2015 Oct;110(10):1686-7. doi: 10.1111/add.13018.

3. Farsalinos K, Voudris V, Poulas K. Response to Shihadeh et al. (2015): E-cigarettes generate high levels of aldehydes only in 'dry puff' conditions. Addiction. 2015 Nov;110(11):1862-4. doi: 10.1111/add.13078.

4. West R. Conflicts of conscience in Addiction. Addiction. 2015 Nov;110(11):1864. doi: 10.1111/add.13069.

5. Farsalinos KE, Voudris V, Poulas K. E-cigarettes generate high levels of aldehydes only in 'dry puff' conditions. Addiction. 2015 Aug;110(8):1352-6.

6. Farsalinos KE, Romagna G, Tsiapras D, Kyrzopoulos S, Voudris V. Evaluation of electronic cigarette use (vaping) topography and estimation of liquid consumption: implications for research protocol standards definition and for public health authorities' regulation. Int J Environ Res Public Health. 2013 Jun 18;10(6):2500-14. doi: 10.3390/ijerph10062500.

On 2016 Oct 18, Clive Bates commented:

Dr. Farsalinos has provided a full disclosure statement. It would be better if the authors or their supporters responded to substantive criticisms of their work than complaining about who is making them.

This comment suggests that criticising other papers is somehow an 'attack'. Normally we refer to evidence-based criticism as 'science'.

With all due respect, the criticism of Jensen RP, 2015 by Farsalinos and I in a letter published in Addiction is well-founded. No adequate response to these criticisms has been made or will be. The reason is that the letter points to fundamental flaws in the study design: operating the device at excessively high temperature, and thereby running in 'dry puff' conditions that no human user ever experiences for more than an instant. Though completely unrealistic for humans, laboratory machines will reliably, but misleadingly, measure emissions under these conditions for hours.

The tendency among researchers and journals unfamiliar with actual vaping and vapers, but keen to publish laboratory results, is to overlook these real-world phenomena. That is why responsible researchers in the field need to know what methodology was used so they can decide whether the results are possibly interesting or whether they should be discarded as another failure of scientific comprehension, unchecked by peer review.

On 2016 Sep 14, Hong Yin commented:

Since Konstantinos Farsalinos had unrestricted funds from electronic cigarette companies, a full disclosure of the relationship to electronic cigarette industry is needed before making this kind of judgmental comment.

Similar attack was made on another published paper in The New England Journal of Medicine entitled “Hidden formaldehyde in e-cigarette aerosols” (1, 2).

In his comment, Farsalinos cited his recent publication “E-cigarettes generate high levels of aldehydes only in 'dry puff' conditions” (3). This paper received comments such as “Insufficient Method Description”, “Incomplete Result Reporting”, and “Unjustified Conclusions" (4) (quoted from the commentary).

References:

1. Jensen RP, Luo W, Pankow JF, Strongin RM, Peyton DH. Hidden formaldehyde in e-cigarette aerosols. N Engl J Med. 2015 Jan 22;372(4):392-4.

2. Bates CD, Farsalinos KE. Research letter on e-cigarette cancer risk was so misleading it should be retracted. Addiction. 2015 Oct;110(10):1686-7.

3. Farsalinos KE, Voudris V, Poulas K. E-cigarettes generate high levels of aldehydes only in 'dry puff' conditions. Addiction. 2015 Aug;110(8):1352-6.

4. Shihadeh A, Talih S, Eissenberg T. Commentary on Farsalinos et al. (2015): E-cigarettes generate high levels of aldehydes only in 'dry puff' conditions. Addiction. 2015 Nov;110(11):1861-2.

Disclosure: I have nothing to disclose with any cigarette industry.

On 2016 Jul 14, Clive Bates commented:

It is disappointing that the only reaction to Farsalinos' comment so far (anonymously via PubPeer) has been in relation to a disclosure statement, which in any event he has now added. There has been no substantive criticism of his work or response to his critique. Perhaps now the discussion should return to addressing the multiple methodological weaknesses he has identified in the Ji et al study.

Farsalinos and colleagues' study at ref #2 above shows clearly how machine measurements of e-cigarettes can become a misleading proxy for human exposure as power increases. Laboratory machines lack a critical control feedback - the reaction to a harsh taste by rapidly stopping vaping - that human subjects have. The acrid taste arises in situations where the liquid is undergoing thermal decomposition and its chemistry changes. If this is the case in the present experiment, the measurements are made under conditions that create exposures no human will experience. If Ji et al did not control to prevent this 'dry puff' condition arising during their experiment, the paper does not offer a basis for making any human risk assessments and is easily open to misinterpretation.

The uncertainty over the magnitude of the implied exposure, the lack of comparative context (the obvious comparator being tobacco smoke) and an uncertain operating regime capable of creating completely unrealistic exposures are major deficiencies. No conclusions about e-cigarette use and human health should be drawn from this study and the authors should discourage any over-interpretation of their findings.

Disclosure: I am a longstanding advocate for evidence-based 'harm reduction' approaches to public health. I was director of Action on Smoking and Health UK from 1997-2003. I have no competing interests with respect to any of the relevant industries.

On 2016 Jul 06, Konstantinos Farsalinos commented:

Paracelsus, the herald of modern toxicology, mentioned in the 16th century that: nothing is a poison and everything is poisonous; solely the dose determines that a thing is not a poison (Sola dosis facit venenum) (1). This principle, which still stands today and forms the basis of the dose-response concept, has been largely ignored by many scientists including the authors of this study. This study represents an unfortunate case of complete failure of the peer-review and editorial process of the journal.

One of the basic prerequisites for accepting a manuscript for publication is to provide a clear and detailed presentation of the methodology used in the experimental setup. In this case, the authors provide no information about:

1. The electronic cigarette device used in the experiment

2. The power settings on the electronic cigarette device that were used in the experiment

3. The puffing patterns (puff duration and interpuff interval). A range of 2 to 5s puff duration was mentioned only for the assessment of particle number concentration but not for cell exposure

4. The number of puffs performed during the aerosol generation procedure

5. The amount of liquid consumed during the aerosol generation procedure

6. The amount of aerosol dilution into the cell medium. There is no mention of the amount of cell medium used in the impingers and the amount of aerosol that was diluted into the cell medium.

7. The authors cite a study which has not been published or even accepted for publication but is marked as "submitted" for further details on the methodology (reference 11 of the manuscript)

Omissions 1-3 are extremely important because it is well-known that electronic cigarettes can be easily abused in a laboratory setting, which may result in the evaluation of unrealistic, dry puff, conditions. Such conditions are irrelevant to human exposure (2).

Omissions 4-6 are extremely important because they violate the basic principle of toxicology which dictates that the dose determines the toxicity. In fact, it is very easy in a cell culture to create an adverse cell response, just by manipulating the dose of exposure. Therefore, it is crucial to present the level (dose) of exposure and ensure that this is relevant to human exposure. Otherwise, the findings cannot be extrapolated to human effects and can only be used for comparing two different exposures. For the latter, it is surprising that the authors did not attempt to perform the same experiment with the same conditions and the same level of exposure with tobacco cigarette smoke. Thus, the study findings cannot even be used for comparative purposes.

Omission 7 is really unprecedented. It is the first time that i see a reference to a non-existing publication (marked as "submitted", so, not reviewed and not accepted) being accepted by the reviewers and the editor.

These omissions raise some important issues about the peer-review process and the editorial assessment of that study. In my opinion, this is a typical example of failure of the peer-review process and the study should be retracted or revised.

Disclosure. No funding or other support was provided for this comment. The author has no financial or other interest on e-cigarette companies. Two studies (unpublished) were performed with unrestricted funds provided to the institution (Onassis Cardiac Surgery Center) by 2 electronic cigarette companies in 2013, for which no researcher (including the author of this comment) received any financial compensation. Funding was provided more than 3 years ago, so the declaration is not necessary based on the ICMJE guidelines. Two published studies were performed using funds provided by the non-profit association AEMSA in 2013 (less than 3 years ago) and one published study was funded by the non-profit association Tennessee Smoke-Free Association in 2015.

References

1. Borzelleca JF. Paracelsus: herald of modern toxicology. Toxicol Sci. 2000 Jan;53(1):2-4.

2. Farsalinos KE, Voudris V, Poulas K. E-cigarettes generate high levels of aldehydes only in 'dry puff' conditions. Addiction. 2015 Aug;110(8):1352-6.

On 2016 Jul 13, Jenna Wong commented:

None

On 2016 Jul 16, David Keller commented:

Doxepin - an antidepressant with strong scientific evidence supporting its off-label use for insomnia

Wong wrote: "Regarding the off-label use of antidepressants for insomnia, we disagree with Keller that it is supported by strong scientific evidence".

To demonstrate that Wong is incorrect, I cite doxepin, an antidepressant with strong evidence supporting its off-label use for insomnia at the 10 mg dose approved only for depression. Both of the references cited by Wong to contradict me were published in 2005, 5 years before the FDA approval of doxepin for insomnia. In 2010, the FDA approved doxepin for insomnia at the doses of 3 mg and 6 mg, which can be given sequentially if needed. Physicians continue to prescribe cheap generic doxepin 10 mg to treat insomnia (off-label at 10 mg), based on the strong scientific evidence supporting its FDA-approved insomnia doses of 3 mg and 6 mg. Doxepin 10 mg is the lowest dose approved for use as a cheap generic antidepressant, and I daresay that the vast majority of prescriptions for that dose are written off-label for insomnia, which has the benefit of saving the patient substantial money on the high cost of the doses branded for insomnia.

On 2016 Jul 13, Jenna Wong commented:

We thank Keller for his comment. We agree that the level of supporting scientific evidence is the most important aspect to consider when determining the appropriateness of off-label drug use. In our article, we state that our findings “highlight the need to evaluate the evidence supporting off-label antidepressant use,” which neither assumes that all off-label antidepressant use is inappropriate, nor assumes that all off-label indications require official regulatory approval. In cases where off-label use lacks scientific evidence, our position is that physicians should exercise caution and carefully consider the potential risk-benefit ratio when prescribing antidepressants for off-label indications. However, to assess risk-benefit, physicians must have easy access to sources of knowledge that evaluate and summarize the existing scientific evidence.

Regarding the off-label use of antidepressants for insomnia, we disagree with Keller that it is supported by strong scientific evidence. In fact, the article [1] that he cites to support this claim states that “[the] NIH State of Science Conference on insomnia treatments reviewed the status of antidepressants reporting minimal scientific evidence supporting their use,” and that “[of] particular concern in using antidepressants for insomnia are a number of unique potential side effects”. Other review articles have also made similar conclusions [2].

References:

[1] Asnis GM, Thomas M, Henderson MA. Pharmacotherapy Treatment Options for Insomnia: A Primer for Clinicians. Int J Mol Sci. 2015;17(1).

[2] WB M. A review of the evidence for the efficacy and safety of trazodone in insomnia. J Clin Psychiatry. 2005;66(4):469-476.

On 2016 Jun 29, David Keller commented:

In Defense of Off-Label Prescribing of Antidepressants

Wong and colleagues report that, in their study, 29.4% of all antidepressant prescriptions were written "for non-depressive indications, including unapproved (off-label) indications that have not been evaluated by regulatory agencies" [1]. The authors imply that patients treated for unapproved indications with antidepressants would benefit if these off-label treatments were evaluated by regulatory agencies. However, the small profit margins of generic medications make the cost of regulatory approval for every beneficial indication prohibitively expensive.

Recently, a study by Eguale and colleagues demonstrated that off-label use of prescription drugs is associated with increased adverse drug events only when such use lacks strong scientific evidence [2]. Wong and colleagues found that the two most common off-label indications for antidepressants were insomnia and pain [1], both of which are supported by strong scientific evidence [3], [4]. Less common off-label indications, such as migraine, also have strong scientific support [5].

The findings of Eguale and colleagues demonstrate that patient safety does not require specific regulatory approval for every off-label indication, and that it is not necessary to spend the large sums of money which would be required to obtain FDA approval for every off-label indication. [6]

References

1: Wong J, Motulsky A, Eguale T, Buckeridge DL, Abrahamowicz M, Tamblyn R. Treatment Indications for Antidepressants Prescribed in Primary Care in Quebec, Canada, 2006-2015. JAMA.2016;315(20):2230-2232. doi:10.1001/jama.2016.3445.

2: Eguale T, Buckeridge DL, Verma A, Winslade NE, Benedetti A, Hanley JA, Tamblyn R . Association of Off-label Drug Use and Adverse Drug Events in an Adult Population. JAMA Intern Med. 2016 Jan 1;176(1):5563. doi:10.1001/jamainternmed.2015.6058. PubMed PMID: 26523731.

3: Asnis GM, Thomas M, Henderson MA. Pharmacotherapy Treatment Options for Insomnia: A Primer for Clinicians. Int J Mol Sci. 2015 Dec 30;17(1). pii: E50. doi: 10.3390/ijms17010050. PubMed PMID: 26729104; PubMed Central PMCID:PMC4730295.

4: Janakiraman R, Hamilton L, Wan A. Unravelling the efficacy of antidepressants as analgesics. Aust Fam Physician. 2016 Mar;45(3):113-7. PubMed PMID: 27052046.

5: Jackson JL, et al. A Comparative Effectiveness Meta-Analysis of Drugs for the Prophylaxis of Migraine Headache. PLoS One. 2015 Jul 14;10(7):e0130733. doi:10.1371/journal.pone.0130733. eCollection 2015. PubMed PMID: 26172390; PubMed Central PMCID: PMC4501738.

6: Keller DL. In Defense of Off-label Prescribing. JAMA Intern Med. 2016 Jun 1;176(6):861. doi: 10.1001/jamainternmed.2016.1403. PubMed PMID: 27273485.