On 2016 Jun 02, David Keller commented:

Information Theory versus FDA regulations governing colon cancer screening

The ColoGuard colorectal cancer screening test is composed of 7 different sub-tests, including an assay for occult fecal hemoglobin, 5 assays for specific DNA changes associated with malignancy, and an assay for normal DNA to standardize the sample. The results of these seven subtests are combined using complex equations to yield a Composite Score ("CS") ranging from 0 to 1000 and directly related to the likelihood of colon neoplasia. A CS of 183 was established as the threshold for further screening with colonoscopy; If the CS is larger than or equal to 183, the patient is sent for colonoscopy, and if the CS is lower than 183, the patient is spared further testing. [1,2]

The FDA prohibits the release of any of the test data generated by the ColoGuard system except the single bit of information in the final result: positive or negative, over or under the threshold, yes or no to colonoscopy, 1 or 0. This one bit of information represents the distillation of the combined results from the seven sub-tests, but much of the information in those test results is lost in the process of determining the value of that single bit.

Now, consider the composite score, which can vary from 0 to 1000. How many bits of information does the CS convey? The answer is nearly 10 bits of information, because 10 bits can represent 0 through 1023 in decimal numeral, or 000000000 through 111111111 in binary. So, we start with a composite score which requires 10 bits of information to quantify the patient's risk of colon cancer, and we finish by distilling the result down to a single bit of information: positive or negative. In so doing, we have thrown away 9 perfectly good bits of information.

A negative ColoGuard result conveys just one bit of information about the status of the patient's colon epithelium, informing us only that the composite score was in the range of 0 to 182, a range which requires almost 8 more bits to specify precisely. The FDA forces Exact Sciences to throw away almost 7 perfectly good bits of information about the patient's colon epithelium. These discarded 7 bits tell the patient whether his composite score was 0 or 182 or somewhere in between, given an overall negative screening result.

The purpose of the FDA regulation is to force clinicians to adhere strictly to the decision threshold for colonoscopy of 183, which has been validated in a large clinical trial. But what of the patient who develops "soft" or borderline indications for colonoscopy some time later. The clinician may be undecided whether the patient's borderline signs and symptoms warrant a diagnostic colonoscopy or not. Knowing that the patient's recent composite score had been 17 or 170 could help tip the decision one way or the other.

We may learn over time that composite scores can exhibit informative trends over time. What is the significance of a composite score rising from 15 to 160 over 3 years? We will never know if we throw away the 7 bits that let us quantify the composite score with precision.

We have discussed the loss of clinical information which occurs when the ColoGuard composite score is distilled down to one bit. There is also substantial loss of information when the Cologuard sub-test results are crushed together to derive the composite score. In my next comment, I will discuss some effects of the loss of information which occurs related to ColoGuard's fecal hemoglobin concentration sub-test. [3] We will learn how the FDA's prohibition of patient access to Cologuard's internal sub-test results could be harmful to patients.

References

1: Imperiale TF, Ransohoff DF, Itzkowitz SH, Turnbull BA, Ross ME; Colorectal Cancer Study Group. Fecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population. N Engl J Med. 2004 Dec 23;351(26):2704-14. PubMed PMID: 15616205.

2: Imperiale TF et al. Online Supplement to the above print article, accessed on 4/18/2016. http://www.nejm.org/doi/suppl/10.1056/NEJMoa1311194/suppl_file/nejmoa1311194_appendix.pdf

3: PubMed Commons comment: http://www.ncbi.nlm.nih.gov/pubmed/27235008#cm27235008_15979

On 2016 Jun 02, David Keller commented:

Information Theory versus FDA regulations governing colon cancer screening

The ColoGuard colorectal cancer screening test is composed of 7 different sub-tests, including an assay for occult fecal hemoglobin, 5 assays for specific DNA changes associated with malignancy, and an assay for normal DNA to standardize the sample. The results of these seven subtests are combined using complex equations to yield a Composite Score ("CS") ranging from 0 to 1000 and directly related to the likelihood of colon neoplasia. A CS of 183 was established as the threshold for further screening with colonoscopy; If the CS is larger than or equal to 183, the patient is sent for colonoscopy, and if the CS is lower than 183, the patient is spared further testing. [1,2]

The FDA prohibits the release of any of the test data generated by the ColoGuard system except the single bit of information in the final result: positive or negative, over or under the threshold, yes or no to colonoscopy, 1 or 0. This one bit of information represents the distillation of the combined results from the seven sub-tests, but much of the information in those test results is lost in the process of determining the value of that single bit.

Now, consider the composite score, which can vary from 0 to 1000. How many bits of information does the CS convey? The answer is nearly 10 bits of information, because 10 bits can represent 0 through 1023 in decimal numeral, or 000000000 through 111111111 in binary. So, we start with a composite score which requires 10 bits of information to quantify the patient's risk of colon cancer, and we finish by distilling the result down to a single bit of information: positive or negative. In so doing, we have thrown away 9 perfectly good bits of information.

A negative ColoGuard result conveys just one bit of information about the status of the patient's colon epithelium, informing us only that the composite score was in the range of 0 to 182, a range which requires almost 8 more bits to specify precisely. The FDA forces Exact Sciences to throw away almost 7 perfectly good bits of information about the patient's colon epithelium. These discarded 7 bits tell the patient whether his composite score was 0 or 182 or somewhere in between, given an overall negative screening result.

The purpose of the FDA regulation is to force clinicians to adhere strictly to the decision threshold for colonoscopy of 183, which has been validated in a large clinical trial. But what of the patient who develops "soft" or borderline indications for colonoscopy some time later. The clinician may be undecided whether the patient's borderline signs and symptoms warrant a diagnostic colonoscopy or not. Knowing that the patient's recent composite score had been 17 or 170 could help tip the decision one way or the other.

We may learn over time that composite scores can exhibit informative trends over time. What is the significance of a composite score rising from 15 to 160 over 3 years? We will never know if we throw away the 7 bits that let us quantify the composite score with precision.

We have discussed the loss of clinical information which occurs when the ColoGuard composite score is distilled down to one bit. There is also substantial loss of information when the Cologuard sub-test results are crushed together to derive the composite score. In my next comment, I will discuss some effects of the loss of information which occurs related to ColoGuard's fecal hemoglobin concentration sub-test. [3] We will learn how the FDA's prohibition of patient access to Cologuard's internal sub-test results could be harmful to patients.

References

1: Imperiale TF, Ransohoff DF, Itzkowitz SH, Turnbull BA, Ross ME; Colorectal Cancer Study Group. Fecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population. N Engl J Med. 2004 Dec 23;351(26):2704-14. PubMed PMID: 15616205.

2: Imperiale TF et al. Online Supplement to the above print article, accessed on 4/18/2016. http://www.nejm.org/doi/suppl/10.1056/NEJMoa1311194/suppl_file/nejmoa1311194_appendix.pdf

3: PubMed Commons comment: http://www.ncbi.nlm.nih.gov/pubmed/27235008#cm27235008_15979