On 2013 Jul 05, Richard Simon commented:

This is an interesting and important paper but some have misinterpreted it’s findings. The authors addressed the prognostic significance of published gene expression signatures for breast cancer (mostly for non-metastatic disease). They demonstrated that prognostic signatures can be developed over 90% of the time from random sets of >100 genes and that the signatures generated from sets of random genes are often as prognostic as published gene expression signatures. The reason is that there are an enormous number of genes that are correlated with cell proliferation and cell proliferation is strongly correlated with prognosis (estrogen receptor expression is strongly associated with outcome and prognosis and there are thousands of estrogen receptor target genes). Most published gene expression signatures are no longer prognostic after adjustment for the proliferation meta-gene. The take home message is that authors of biological mechanism papers should not claim that the genes they discovered in an experimental model system have relevance for the human breast cancer by showing that a signature based on their genes are prognostic in human breast cancer. That claim is common in cancer biology. The paper should not be misinterpreted to mean that claims of the prognostic accuracy of gene expression signatures are erroneous or that such signatures are not potentially useful for medical decision making. In fact, there are well documented pitfalls in the evaluation of predictive accuracy of prognostic signatures (e.g. using the same dataset to develop the signature and to evaluate it without using complete cross-validation). Predictive signatures which identify patients most likely to benefit from a specific treatment tend to be more useful than prognostic signatures derived based on a heterogeneous collection of cases. Nevertheless, prognostic signatures can be therapeutically relevant. This paper is well done however and it’s conclusions are carefully drawn.

On 2013 Jul 05, Richard Simon commented:

This is an interesting and important paper but some have misinterpreted it’s findings. The authors addressed the prognostic significance of published gene expression signatures for breast cancer (mostly for non-metastatic disease). They demonstrated that prognostic signatures can be developed over 90% of the time from random sets of >100 genes and that the signatures generated from sets of random genes are often as prognostic as published gene expression signatures. The reason is that there are an enormous number of genes that are correlated with cell proliferation and cell proliferation is strongly correlated with prognosis (estrogen receptor expression is strongly associated with outcome and prognosis and there are thousands of estrogen receptor target genes). Most published gene expression signatures are no longer prognostic after adjustment for the proliferation meta-gene. The take home message is that authors of biological mechanism papers should not claim that the genes they discovered in an experimental model system have relevance for the human breast cancer by showing that a signature based on their genes are prognostic in human breast cancer. That claim is common in cancer biology. The paper should not be misinterpreted to mean that claims of the prognostic accuracy of gene expression signatures are erroneous or that such signatures are not potentially useful for medical decision making. In fact, there are well documented pitfalls in the evaluation of predictive accuracy of prognostic signatures (e.g. using the same dataset to develop the signature and to evaluate it without using complete cross-validation). Predictive signatures which identify patients most likely to benefit from a specific treatment tend to be more useful than prognostic signatures derived based on a heterogeneous collection of cases. Nevertheless, prognostic signatures can be therapeutically relevant. This paper is well done however and it’s conclusions are carefully drawn.