Smith, N. (2020, April 18). Taiwan’s Vice-President Chen Chien-jen on his country’s fight with Covid-19. The Telegraph. https://www.telegraph.co.uk/global-health/science-and-disease/taiwans-vice-president-chen-chien-jen-countrys-fight-covid-19/

Take Away: The exact frequency of false positive test results for COVID-19 is unknown. Real world data on COVID-19 testing suggests that rigorous testing regimes likely produce fewer than 1 in 10,000 (<0.01%) false positives, orders of magnitude below the frequency proposed here.

The Claim: The reported numbers for new COVID-19 cases are overblown due to a false positive rate of 0.8%

The Evidence: In this opinion article, the author correctly conveys the concern that for large testing strategies, case rates could become inflated if there is (a) a high false positive rate for the test and (b) there is a very low prevalence of the virus within the population. The false positive rate proposed by the author is 0.8%, based on the "lowest value" for similar tests given by a briefing to the UK's Scientific Advisory Group for Emergencies (1).

In fact, the briefing states that, based on another analysis, among false positive rates for 43 external quality assessments, the interquartile range for false positive rate was 0.8-4.0%. The actual lowest value for false positive rate from this study was 0% (2).

An upper limit for false positive rate can also be estimated from the number of tests conducted per confirmed COVID-19 case. In countries with low infection rates that have conducted widespread testing, such as Vietnam and New Zealand, at multiple periods throughout the pandemic they have achieved over 10,000 tests per positive case (3). Even if every single positive was false, the false positive rate would be below 0.01%.

The prevalence of the virus within a population being tested can affect the positive predictive value of a test, which is the likelihood that a positive result is due to a true infection. The author here assumes the current prevalence of COVID-19 in the UK is 1 in 1,000 and the expected rate of positive results is 0.1%. Data from the University of Oxford and the Global Change Data Lab show that the current (Sept. 22, 2020) share of daily COVID-19 tests that are positive in the UK is around 1.7% (4). Therefore, based on real world data, the probability that a patient is positive for the test and does have the disease is 99.4%.

Sources: (1) https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/895843/S0519_Impact_of_false_positives_and_negatives.pdf

(2) https://www.medrxiv.org/content/10.1101/2020.04.26.20080911v3.full.pdf+html

(3) https://ourworldindata.org/coronavirus-data-explorer?yScale=log&zoomToSelection=true&country=USA~DEU~IND~ITA~AUS~VNM~FIN~NZL~GBR&region=World&testsPerCaseMetric=true&interval=smoothed&aligned=true&smoothing=7&pickerMetric=location&pickerSort=asc

(4) https://ourworldindata.org/coronavirus-data-explorer?zoomToSelection=true&country=USA~DEU~IND~ITA~AUS~VNM~FIN~NZL~GBR&region=World&positiveTestRate=true&interval=smoothed&aligned=true&smoothing=7&pickerMetric=location&pickerSort=asc

Take Away: This is an incorrect usage of the term "false positive." A positive PCR test result from a recovered infection is a valid and true positive.

Claim: PCR tests for SARS-CoV-2 give false positive results when there is no active virus.

Evidence: The diagnostic PCR tests currently in widespread use are designed to detect the presence of the SARS-CoV-2 viral RNA in a clinical sample. The RNA is only a part of the complete virus and is not infectious on its own. Research has shown that viral RNA can be detected in some samples up to 12 weeks after onset of symptoms (1). In other words, this is like testing if an oven is warmer than the room temperature - it could be hot even after it has been turned off.

By definition, in the context of SARS-CoV-2 PCR tests, a "false positive" means that a test result is deemed positive when in reality there was no viral RNA in the sample. If a person is recovering from an infection, gets tested, and then is given a positive test result, that is a true positive regardless of whether they are infectious or not.

Sources: 1) https://www.cdc.gov/coronavirus/2019-ncov/hcp/duration-isolation.html

Take away: People are infectious for only part of the time they test positive. The tests for COVID-19 were granted emergency status by the FDA so some debate concerning the most ideal number of cycles is to be expected. It is worth noting that the FDA has the disclaimer "Negative results do not preclude 2019-nCoV infection and should not be used as the sole basis for treatment or other patient management decisions. Negative results must be combined with clinical observations, patient history, and epidemiological information (2)."

The claim: Up to 90 percent of people diagnosed with coronavirus may not be carrying enough of it to infect anyone else

The evidence: Per Walsh et al. (1), SARS-CoV-2 virus (COVID-19) is most likely infectious if the number of PCR cycles is <24 and the symptom onset to test is <8 days. RT-PCR detects the RNA, not the infectious virus. Therefore, setting the cycle threshold at 37-40 cycles will most likely result in detecting some samples with virus which is not infectious. As the PCR tests were granted emergency use by the FDA (samples include 2-9), it is not surprising that some debate exists currently about where the cycle threshold should be. Thresholds need to be set and validated for dozens of PCR tests currently in use. If identifying only infectious individuals is the goal, a lower cycle number may be justified. If detection of as many cases as possible to get closer to the most accurate death rate is the goal, setting the cycle threshold at 37-40 makes sense. A lower threshold will result in fewer COVID-19 positive samples being identified. It is worth noting that the emergency use approval granted by the FDA includes the disclaimer that a negative test does not guarantee that a person is not infected with COVID-19. RNA degrades easily. If samples are not kept cold or properly processed, the virus can degrade and result in a false negative result.

Source: 1 https://academic.oup.com/cid/advance-article/doi/10.1093/cid/ciaa638/5842165

2 https://www.fda.gov/media/134922/download

3 https://www.fda.gov/media/138150/download

4 https://www.fda.gov/media/137120/download

5 https://www.fda.gov/media/136231/download

6 https://www.fda.gov/media/136472/download

7 https://www.fda.gov/media/139279/download

8 https://www.fda.gov/media/136314/download

9 https://www.fda.gov/media/140776/download

Take Away: Diagnostic tests are most useful when they are both sensitive and rapid. The sensitivity of SARS-CoV-2 PCR tests is not the issue, but rather the time it takes to get a result. Additionally, the "90%" statistic is likely misleading due to the data source and not generalisable to all testing results.

The Claim: The usual PCR diagnostic tests may be too sensitive and too slow, with up to 90% of positive cases due to trace amounts of virus.

The Evidence: Polymerase Chain Reaction (PCR)-based tests, which are currently in the most widespread use for detection of SARS-CoV-2 RNA, involves a molecular process that amplifies target DNA sequences in repeated temperature-dependent cycles. The amount of target DNA is measured after each cycle and the number of the cycle when the target can be reliably detected is often referred to as the cycle threshold (Ct). The Ct value is proportional to the amount of starting DNA in the sample and can be used to estimate the viral load of a patient. In some ways this is like a teacher making photocopies of a chapter from a textbook until they have enough for all their students.

However, Ct values are relative measurements and need to be directly compared to controls for every sample - a Ct value taken alone can be meaningless. For instance, consider an infected patient who is tested twice: the first time they are gently swabbed and the sample is relatively dilute, the second time they are vigorously swabbed and the sample is relatively concentrated. The resulting Ct values could be drastically different. Therefore, Ct values need to be considered carefully in the proper context for making medical or policy decisions. The FDA also recommends that a PCR result alone should not be used to determine infection status.

Positive results are indicative of the presence of SARS-CoV-2 RNA; clinical correlation with patient history and other diagnostic information is necessary to determine patient infection status. (1)

Current PCR test results are generally given as a binary positive/negative based on a cutoff value for Ct. The cutoff needs to be determined based on the performance of each individually developed SARS-CoV-2 test, of which there are currently over 160 that have been granted emergency use authorization by the FDA (2). Based on unpublished data from the CDC, setting a stringent Ct cutoff of 30 could return negative results in patients who are both infected and potentially infectious (3 Fig 5). Furthermore, a 30 cycle cutoff would return invalid results for samples which are too diluted. Based on the same CDC data, up to 30% of potentially infectious patients would get invalid results and need to be re-swabbed, thereby extending the time between getting infected and getting a positive result.

The period of time when RNA from SARS-CoV-2 can be detected (and a positive PCR test result returned) may extend up to 12 weeks after recovery, with Ct values trending higher over time (3,4). According to The New York Times article, they looked at Ct values from people who tested positive in Massachusetts in July and found 85-90% of results had Ct values greater than 30. The epidemiology of COVID-19 is highly time and region dependent. Massachusetts had a peak in COVID-19 hospitalizations on April 21 (5), which is 9-12 weeks prior to the testing data analyzed by The NY Times. Therefore, the detection of a large proportion of people with lingering viral RNA is not surprising. These results are likely not universal and can not be applied to other regions, especially where community spread is still significant.

Sources:

(1) https://www.fda.gov/media/135900/download

(2) https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas

(3) https://www.cdc.gov/coronavirus/2019-ncov/hcp/duration-isolation.html

(4) Li N, Wang X, Lv T. Prolonged SARS-CoV-2 RNA Shedding: Not a Rare Phenomenon. J Med Virol 2020 Apr 29. doi: 10.1002/jmv.25952.

(5) https://www.bostonherald.com/2020/05/22/massachusetts-finally-seeing-downward-coronavirus-trends/

Take away: Current polymerase chain reaction (PCR) testing technology is very sensitive and specific. Even for rapidly developed new tests for the novel coronavirus, SARS-CoV-2, available clinical data indicates they are highly accurate.

The claim: SARS-CoV-2 testing is unreliable and plagued by false positive results.

The evidence: Any diagnostic test has some degree of error that is typically very low for FDA approved products. For SARS-CoV-2 tests, which detect the presence of the virus that causes COVID-19, although rare, it is possible to get a positive result when you may not have been exposed or infected by the virus. In other words, a false positive. So how frequently do false positives occur?

There is no universal false positive rate for SARS-CoV-2 test results because there are dozens of different tests that have been developed and deployed, each with their own error rate. As of August 26, 2020, there are 146 commercial diagnostic tests that have received emergency use authorization from the FDA. Data from clinical performance testing submitted to the FDA indicates that PCR tests are highly accurate. For example, the specific PCR test mentioned by the author, Quest Diagnostics SARS-CoV-2 rRT-PCR, obtained 100% correct results in clinical evaluation studies (n = 60), and 100% true negative results in a random population of samples from before the pandemic (n = 72).

Additional considerations: In addition to PCR technology-based tests, which detect the viral RNA genome and require lab processing, there are antigen tests, which use antibodies to detect viral proteins and can be rapidly performed in point-of-care settings. Antigen tests are much easier to perform than PCR tests, but they can be less sensitive. For example, the LumiraDx SARS-CoV-2 Ag Test, when compared to PCR, has an overall agreement of 96.9%.

The author provides two anecdotes of high-profile personnel who obtained false positive test results. For the Ohio Governor, his initial positive was from an antigen test, not a PCR test. The NFL quarterback is part of a unique population that is presumed to be largely SARS-CoV-2 negative but is being tested frequently and repeatedly. This scenario increases the probability that a positive test result may be false. However, the NFL in early August said it has conducted over 75,000 tests, so unless there are many additional cases of false positives, this suggests that their testing methodology is over 99.99% accurate.

Herper, M. (2020, July 1). Covid-19 vaccine from Pfizer and BioNTech shows positive results. CNBC. https://www.cnbc.com/2020/07/01/coronavirus-vaccine-from-pfizer-and-biontech-shows-positive-results-report-says.html

Take away: COVID-19 infections result in antibodies in almost all cases. These antibodies probably give immunity to future infection for at least some time, although how long is still not known.

The claim: The presence of antibodies to SARS-CoV2 does not guarantee future immunity from future COVID-19 infection.

The evidence: COVID-19 has not been present in the human population long enough to know how long immunity will last. There is some evidence to suggest that having COVID-19 typically leads to antibodies will provide at least some immunity to future infections. The vast majority (>90%) of serious (1-3) and mild (4,5) COVID-19 infections do result in the production of antibodies and it has been found that neutralizing antibodies provide immunity to reinfection in monkeys (6). We do not know how long immunity lasts. The best evidence is from the related coronavirus infections SARS and MERS. SARS and MERS infections result in antibodies that last for at least 1-3 years (7-9).

Source:

1. https://academic.oup.com/cid/advance-article/doi/10.1093/cid/ciaa344/5812996
2. https://erj.ersjournals.com/content/early/2020/05/13/13993003.00763-2020.abstract
3. https://www.nature.com/articles/s41591-020-0897-1)
4. https://www.sciencedirect.com/science/article/pii/S2352396420302905
5. https://www.medrxiv.org/content/10.1101/2020.07.11.20151324v1
6. https://www.biorxiv.org/content/10.1101/2020.03.13.990226v2.abstract
7. https://www.jimmunol.org/content/jimmunol/181/8/5490.full.pdf
8. https://wwwnc.cdc.gov/eid/article/13/10/07-0576_article,
9. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5512479/

Vogels, C. B. F., Brackney, D., Wang, J., Kalinich, C. C., Ott, I., Kudo, E., Lu, P., Venkataraman, A., Tokuyama, M., Moore, A. J., Muenker, M. C., Casanovas-Massana, A., Fournier, J., Bermejo, S., Campbell, M., Datta, R., Nelson, A., Team, Y. I. R., Cruz, C. D., … Grubaugh, N. (2020). SalivaDirect: Simple and sensitive molecular diagnostic test for SARS-CoV-2 surveillance. MedRxiv, 2020.08.03.20167791. https://doi.org/10.1101/2020.08.03.20167791

Peeling, Rosanna W., Catherine J. Wedderburn, Patricia J. Garcia, Debrah Boeras, Noah Fongwen, John Nkengasong, Amadou Sall, Amilcar Tanuri, and David L. Heymann. ‘Serology Testing in the COVID-19 Pandemic Response’. The Lancet Infectious Diseases 0, no. 0 (17 July 2020). https://doi.org/10.1016/S1473-3099(20)30517-X.

Weinberger, D. M., Chen, J., Cohen, T., Crawford, F. W., Mostashari, F., Olson, D., Pitzer, V. E., Reich, N. G., Russi, M., Simonsen, L., Watkins, A., & Viboud, C. (2020). Estimation of Excess Deaths Associated With the COVID-19 Pandemic in the United States, March to May 2020. JAMA Internal Medicine. https://doi.org/10.1001/jamainternmed.2020.3391

Zhong, H., Wang, Y., Shi, Z., Zhang, L., Ren, H., He, W., Zhang, Z., Zhu, A., Zhao, J., Xiao, F., Yang, F., Liang, T., Ye, F., Zhong, B., Ruan, S., Gan, M., Zhu, J., Li, F., Li, F., … Zhao, J. (2020). Characterization of Microbial Co-infections in the Respiratory Tract of hospitalized COVID-19 patients. MedRxiv, 2020.07.02.20143032. https://doi.org/10.1101/2020.07.02.20143032

Meyer, B., Torriani, G., Yerly, S., Mazza, L., Calame, A., Arm-Vernez, I., Zimmer, G., Agoritsas, T., Stirnemann, J., Spechbach, H., Guessous, I., Stringhini, S., Pugin, J., Roux-Lombard, P., Fontao, L., Siegrist, C.-A., Eckerle, I., Vuilleumier, N., & Kaiser, L. (2020). Validation of a commercially available SARS-CoV-2 serological immunoassay. Clinical Microbiology and Infection, 0(0). https://doi.org/10.1016/j.cmi.2020.06.024

Pollán, M., Pérez-Gómez, B., Pastor-Barriuso, R., Oteo, J., Hernán, M. A., Pérez-Olmeda, M., Sanmartín, J. L., Fernández-García, A., Cruz, I., Larrea, N. F. de, Molina, M., Rodríguez-Cabrera, F., Martín, M., Merino-Amador, P., Paniagua, J. L., Muñoz-Montalvo, J. F., Blanco, F., Yotti, R., Blanco, F., … Villa, A. V. de la. (2020). Prevalence of SARS-CoV-2 in Spain (ENE-COVID): A nationwide, population-based seroepidemiological study. The Lancet, 0(0). https://doi.org/10.1016/S0140-6736(20)31483-5

Sapoval, N., Mahmoud, M., Jochum, M. D., Liu, Y., Elworth, R. A. L., Wang, Q., Albin, D., Ogilvie, H., Lee, M. D., Villapol, S., Hernandez, K., Berry, I. M., Foox, J., Beheshti, A., Ternus, K., Aagaard, K. M., Posada, D., Mason, C., Sedlazeck, F. J., & Treangen, T. J. (2020). Hidden genomic diversity of SARS-CoV-2: Implications for qRT-PCR diagnostics and transmission. BioRxiv, 2020.07.02.184481. https://doi.org/10.1101/2020.07.02.184481

How COVID-19 revealed 3 critical AI procurement blindspots. (n.d.). World Economic Forum. Retrieved June 22, 2020, from https://www.weforum.org/agenda/2020/06/how-covid-19-revealed-3-critical-blindspots-ai-governance-procurement/

Perencevich, E. N., Diekema, D. J., & Edmond, M. B. (2020). Moving Personal Protective Equipment Into the Community: Face Shields and Containment of COVID-19. JAMA. https://doi.org/10.1001/jama.2020.7477

COVID-19 Medical Misinformation Policy—YouTube Help. (n.d.). Retrieved May 26, 2020, from https://support.google.com/youtube/answer/9891785?hl=en#annotations:53lvgpt_EeqmRf84xl6gxA

Scherer, L. D., Valentine, K. D., Patel, N., Baker, S. G., & Fagerlin, A. (2019). A bias for action in cancer screening? Journal of Experimental Psychology. Applied, 25(2), 149–161. https://doi.org/10.1037/xap0000177

Shental, N., Levy, S., Skorniakov, S., Wuvshet, V., Shemer-Avni, Y., Porgador, A., & Hertz, T. (2020). Efficient high throughput SARS-CoV-2 testing to detect asymptomatic carriers. MedRxiv, 2020.04.14.20064618. https://doi.org/10.1101/2020.04.14.20064618

Fenton, N., Hitman, G. A., Neil, M., Osman, M., & McLachlan, S. (2020). Causal explanations, error rates, and human judgment biases missing from the COVID-19 narrative and statistics [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/p39a4

Shweta, F., Murugadoss, K., Awasthi, S., Venkatakrishnan, A., Puranik, A., Kang, M., Pickering, B. W., O’Horo, J. C., Bauer, P. R., Razonable, R. R., Vergidis, P., Temesgen, Z., Rizza, S., Mahmood, M., Wilson, W. R., Challener, D., Anand, P., Liebers, M., Doctor, Z., … Badley, A. D. (2020). Augmented Curation of Unstructured Clinical Notes from a Massive EHR System Reveals Specific Phenotypic Signature of Impending COVID-19 Diagnosis [Preprint]. Infectious Diseases (except HIV/AIDS). https://doi.org/10.1101/2020.04.19.20067660

Adams, E. R., Anand, R., Andersson, M. I., Auckland, K., Baillie, J. K., Barnes, E., Bell, J., Berry, T., Bibi, S., Carroll, M., Chinnakannan, S., Clutterbuck, E., Cornall, R. J., Crook, D. W., Silva, T. D., Dejnirattisai, W., Dingle, K. E., Dold, C., Eyre, D. W., … Sanchez, V. (2020). Evaluation of antibody testing for SARS-Cov-2 using ELISA and lateral flow immunoassays. MedRxiv, 2020.04.15.20066407. https://doi.org/10.1101/2020.04.15.20066407

El-Sadr, W. M., & Justman, J. (2020). Africa in the Path of Covid-19. New England Journal of Medicine, 0(0), null. https://doi.org/10.1056/NEJMp2008193

Gray, N., Calleja, D., Wimbush, A., Miralles-Dolz, E., Gray, A., De-Angelis, M., Derrer-Merk, E., Oparaji, B. U., Stepanov, V., Clearkin, L., & Ferson, S. (2020). “No test is better than a bad test”: Impact of diagnostic uncertainty in mass testing on the spread of Covid-19 [Preprint]. Epidemiology. https://doi.org/10.1101/2020.04.16.20067884

Newton, P. N., Bond, K. C., Adeyeye, M., Antignac, M., Ashenef, A., Awab, G. R., Babar, Z.-U.-D., Bannenberg, W. J., Bond, K. C., Bower, J., Breman, J., Brock, A., Caillet, C., Coyne, P., Day, N., Deats, M., Douidy, K., Doyle, K., Dujardin, C., … Zaman, M. (2020). COVID-19 and risks to the supply and quality of tests, drugs, and vaccines. The Lancet Global Health, S2214109X20301364. https://doi.org/10.1016/S2214-109X(20)30136-4

Government of Canada. (2020). Government of Canada funds 49 additional COVID-19 research projects – Details of the funded projects. Canada.ca. https://www.canada.ca/en/institutes-health-research/news/2020/03/government-of-canada-funds-49-additional-covid-19-research-projects-details-of-the-funded-projects.html

COVID-19. (n.d.). Retrieved April 17, 2020, from https://www.wwtf.at/covid/index.php?lang=EN

based on cell free DNA fragments in blood; DNA methylation patterns and fragment length distributions can inform on organ of origin.