On 2016 Jun 30, Benjamin Haibe-Kains commented:
Our letter describing the fundamental differences between our analysis (Haibe-Kains et al, Nature 2013; http://www.ncbi.nlm.nih.gov/pubmed/24284626) and the GDSC/CCLE reanalysis has been published in F1000Research: http://f1000research.com/articles/5-825/v1
On 2016 Apr 18, John Quackenbush commented:
The authors’ new analyses violate many basic principles of an unbiased assessment of the reproducibility of drug sensitivity predictors across independent data sets. One cannot pick and chose different measures between experiments that measure many different things and claim agreement when the same parameters, measures by both studies are not well correlated. Nor can one arbitrarily replace a subset of data in one study with data from a second study and then compare the two studies. Anyone should realize the fallacy of conclusions derived from such inappropriate analyses.
On 2016 Apr 18, Benjamin Haibe-Kains commented:
The GDSC and CCLE investigators recently reported that their respective studies exhibit reasonable agreement and yield similar molecular predictors of drug response, seemingly contradicting our previous findings (Haibe-Kains B, 2013). Reanalyzing the authors' published methods and results, we found that their analysis failed to account for variability in the genomic data and importantly, compared different drug sensitivity measures from each study, which substantially deviate from our more stringent consistency assessment. The authors’ new analyses violate basic principles of an unbiased assessment of the reproducibility of drug sensitivity predictors across independent datasets.
We submitted a brief response to the GDSC/CCLE paper to Nature, pointing out the errors. Although reviewers agreed that the inappropriate analytical designs should be brought to the attention of the community, the Nature Editors declined to publish our letter. The reason the Editors gave was that this was a specialized discussion subject to personal interpretation. In our opinion, the fundamentals of good analytical design are neither specialized nor subject to interpretation.
Our rejected letter is available on bioRxiv http://biorxiv.org/content/early/2016/04/13/048470. We encourage the GDSC and CCLE investigators to publish their response and we hope that others share their insights on this important issue.
On 2016 Apr 18, Benjamin Haibe-Kains commented:
The GDSC and CCLE investigators recently reported that their respective studies exhibit reasonable agreement and yield similar molecular predictors of drug response, seemingly contradicting our previous findings (Haibe-Kains B, 2013). Reanalyzing the authors' published methods and results, we found that their analysis failed to account for variability in the genomic data and importantly, compared different drug sensitivity measures from each study, which substantially deviate from our more stringent consistency assessment. The authors’ new analyses violate basic principles of an unbiased assessment of the reproducibility of drug sensitivity predictors across independent datasets.
We submitted a brief response to the GDSC/CCLE paper to Nature, pointing out the errors. Although reviewers agreed that the inappropriate analytical designs should be brought to the attention of the community, the Nature Editors declined to publish our letter. The reason the Editors gave was that this was a specialized discussion subject to personal interpretation. In our opinion, the fundamentals of good analytical design are neither specialized nor subject to interpretation.
Our rejected letter is available on bioRxiv http://biorxiv.org/content/early/2016/04/13/048470. We encourage the GDSC and CCLE investigators to publish their response and we hope that others share their insights on this important issue.
On 2016 Apr 18, John Quackenbush commented:
The authors’ new analyses violate many basic principles of an unbiased assessment of the reproducibility of drug sensitivity predictors across independent data sets. One cannot pick and chose different measures between experiments that measure many different things and claim agreement when the same parameters, measures by both studies are not well correlated. Nor can one arbitrarily replace a subset of data in one study with data from a second study and then compare the two studies. Anyone should realize the fallacy of conclusions derived from such inappropriate analyses.
On 2016 Jun 30, Benjamin Haibe-Kains commented:
Our letter describing the fundamental differences between our analysis (Haibe-Kains et al, Nature 2013; http://www.ncbi.nlm.nih.gov/pubmed/24284626) and the GDSC/CCLE reanalysis has been published in F1000Research: http://f1000research.com/articles/5-825/v1