Dr Natalie Shenker on Twitter. (n.d.). Twitter. Retrieved October 13, 2020, from https://twitter.com/DrNShenker/status/1314475759508107265

ReconfigBehSci on Twitter. (n.d.). Twitter. Retrieved October 12, 2020, from https://twitter.com/SciBeh/status/1314991301344014336

Reynolds, M. (2020, October 7). There is no ‘scientific divide’ over herd immunity. Wired UK. https://www.wired.co.uk/article/great-barrington-declaration-herd-immunity-scientific-divide

Science could learn something from this. Science is too far focused on the idealized positive outcomes that it isn't paying attention to the negative outcomes and using that to better define its outline or overall shape. We need to define a scientific opportunity cost and apply it to the negative side of research to better understand and define what we're searching for.

Of course, how can we define a new scientific method (or amend/extend it) to better take into account negative results--particularly in an age when so many results aren't even reproducible?

Take Away: The new scientific paper confirms earlier modeling work and should not be interpreted as a detailed prediction for future deaths due to the ongoing pandemic.

The Claim: "A scientific review of the science behind lockdown concludes the policy was a MISTAKE & will have caused MORE deaths from Covid-19"

The Evidence: The scientific process involves replication and confirmation of experiments and studies. A new paper replicates and expands on an early modeling study of the COVID-19 pandemic in England (1). Their findings support the earlier results. However, there are limitations to the replication paper, which does not accurately reflect the current state of the pandemic response and does not make detailed predictions for a second wave of infections and deaths.

A recent expert response to the paper further explains (2):

"It needs to be stressed that all the simulations assume that interventions are only in place for 3 months (18th April – 18th July) and then completely relaxed. This gives rise to a strange set of scenarios where a second wave is allowed to progress in an uncontrolled manner."

“It is this that leads to the counter-intuitive headline finding “that school closures would result in more overall covid-19 deaths than no school closures” – actually what the authors find is that a short period of intense lock-down (including the closure of schools) leads to a large second wave if it is allowed to run with no controls. To be fair the authors do highlight this in the paper, but it is not in the reported press release." -Prof Matt Keeling, Professor of Populations and Disease, University of Warwick

Sources:

(1) https://www.bmj.com/content/371/bmj.m3588

(2) https://www.sciencemediacentre.org/expert-reaction-to-reanalysis-of-model-used-for-imperial-report-9-and-impact-of-school-closures/

Australian research gets billion-dollar boost in sweeping stimulus budget. (2020). Nature. Retrieved October 09, 2020, from https://www.nature.com/articles/d41586-020-02835-y?utm_source=twt_nnc&utm_medium=social&utm_campaign=naturenews

Take away: COVID-19 death rate is worse than seasonal influenza death rate.

The claim: Coronavirus mortality was over estimated as 10X worse than seasonal influenza to congress due to misclassifying influenza infection fatality rate as a case fatality rate.

The evidence: Comparing infection fatality ratio (IFR) and case fatality ratio (CFR) is an apples to oranges comparison (1). Case fatality ratios present higher death percentages than infection fatality ratios. At the same time, it is important to understand that COVID-19 and seasonal influenza CFR and IFR numbers are rough approximations of reality and the potential for errors exist in all calculations.

The seasonal IFR rate of influenza was overstated in this article. The claim that seasonal influenza IFR and COVID-19 IFR are the same is based on seasonal influenza IFR of 0.1%. Per the WHO report, seasonal influenza “is usually well below 0.1%” (2). This statement was translated into “0.1% or lower” and then “the WHO also reported that 0.1% is the IFR of seasonal influenza, not the CFR of seasonal influenza as reported in the NEJM editorial” (3).

The article is questioning whether COVID-19 is worse than seasonal influenza due to confusion with IFR and CFR. The article overstated influenza IFR to arrive at the conclusion that COVID-19 and seasonal influenza death rates are the same.

Comparison of influenza and COVID-19 deaths:

Influenza CFR = 0.1-0.2%

(Based on CDC data # deaths / # symptomatic cases, 4).

COVID-19 CFR = 2.8%

(In the USA as of 10/6/2020. Includes asymptomatic cases and may therefore be an underestimate of true CFR, 5-6)

It is also important to note that COVID-19 disease is ongoing with the potential for some of the 7,461,206 cases to die from COVID-19 later. Only 2,935,142 cases in the US are reported as recovered as of 10/6/2020.

Even with the inclusion of asymptomatic cases in the death rate calculation for COVID-19, deaths/cases is at least 10X higher than the deaths/cases calculation of symptomatic influenza based on CDC data.

Sources:

1 https://pubmed.ncbi.nlm.nih.gov/32234121/

2 https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200306-sitrep-46-covid-19.pdf?sfvrsn%3d96b04adf_4

3 https://www.cambridge.org/core/journals/disaster-medicine-and-public-health-preparedness/article/public-health-lessons-learned-from-biases-in-coronavirus-mortality-overestimation/7ACD87D8FD2237285EB667BB28DCC6E9/core-reader

4 https://www.cdc.gov/flu/about/burden/index.html#:~:text=While%20the%20impact%20of%20flu,61%2C000%20deaths%20annually%20since%202010

5 https://coronavirus.iowa.gov/pages/case-counts

6 https://coronavirus.jhu.edu/map.html

Cerase, A. (2020). From “good” intuitions to principled practices and beyond: Ethical issues in risk communication. Geological Society, London, Special Publications, 508. https://doi.org/10.1144/SP508-2020-104

Carter, J. (2020, September 29). The American Public Still Trusts Scientists, Says a New Pew Survey. Scientific American. https://www.scientificamerican.com/article/the-american-public-still-trusts-scientists-says-a-new-pew-survey/

Kekecs, Z., Szaszi, B., & Aczel, B. (2020). ECO, an expert consensus procedure for developing robust scientific outputs [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/9gqru

Thacker, P. D. (2020). A few tiny steps towards transparency: How the Sunshine Act shone light on industry’s influence in medicine. BMJ, 370. https://doi.org/10.1136/bmj.m3229

Nagaraj, A., Shears, E., & Vaan, M. de. (2020). Improving data access democratizes and diversifies science. Proceedings of the National Academy of Sciences, 117(38), 23490–23498. https://doi.org/10.1073/pnas.2001682117

Moreau, D., & Gamble, B. (2020). Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations. Psychological Methods, No Pagination Specified-No Pagination Specified. https://doi.org/10.1037/met0000351

Jr, N. L. (2020, September 14). Why Coming Up With Effective Interventions To Address COVID-19 Is So Hard. FiveThirtyEight. https://fivethirtyeight.com/features/why-coming-up-with-effective-interventions-to-address-covid-19-is-so-hard/

Spellman, B. A. (2015). A Short (Personal) Future History of Revolution 2.0. Perspectives on Psychological Science, 10(6), 886–899. https://doi.org/10.1177/1745691615609918

JNOLive | Multiplicity of Randomized Clinical Trials for Coronavirus Disease 2019. (2020, July 14). https://www.youtube.com/watch?v=H-MWPqgLUvA&feature=youtu.be

Vlasceanu, M., & Coman, A. (2020). The Impact of Social Norms on Belief Update [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/gsem6

Max Primbs on Twitter. (n.d.). Twitter. Retrieved September 14, 2020, from https://twitter.com/MaxPrimbs/status/1304516869509066760

Trump officials interfered with CDC reports on Covid-19. (n.d.). POLITICO. Retrieved September 14, 2020, from https://www.politico.com/news/2020/09/11/exclusive-trump-officials-interfered-with-cdc-reports-on-covid-19-412809

Rapid Reviews COVID-19. (n.d.). Rapid Reviews COVID-19. Retrieved September 11, 2020, from https://rapidreviewscovid19.mitpress.mit.edu/

r/BehSciMeta - Comment by u/dawnlxh on ”A completely re-imagined approach to peer review and publishing: PRINCIPIA”. (n.d.). Reddit. Retrieved September 10, 2020, from https://www.reddit.com/r/BehSciMeta/comments/if03sk/a_completely_reimagined_approach_to_peer_review/g4nnuc5

Hack-a-thons to improve the research culture. (n.d.). Google Docs. Retrieved September 9, 2020, from https://docs.google.com/forms/d/e/1FAIpQLScAlSb9XTdXznvI2GrOzsXgRn_ibRFHrDL5acodMnaUzubs2A/viewform?edit_requested=true&usp=embed_facebook

Science, C. for O. (n.d.). TOP Guidelines. Retrieved September 9, 2020, from https://www.cos.io/our-services/top-guidelines

Postdocs in crisis: Science cannot risk losing the next generation. (2020). Nature, 585(7824), 160–160. https://doi.org/10.1038/d41586-020-02541-9

Susan Athey, July 22, 2020. (2020, August 2). https://www.youtube.com/watch?v=hqTOPrUxDzM

Kasy, M. (2020). How to run an adaptive field experiment. Retrieved from https://maxkasy.github.io/home/files/slides/adaptive_field_slides_kasy.pdf

Joshua Salomon on Twitter. (n.d.). Twitter. Retrieved September 7, 2020, from https://twitter.com/SalomonJA/status/1302767010367983616

Merchants of Doubt. (2020). In Wikipedia. https://en.wikipedia.org/w/index.php?title=Merchants_of_Doubt&oldid=950272903

Tea & Talk Podcast. (2020, June 5). The Royal Society of Edinburgh. https://www.rse.org.uk/publication/tea-talk-podcast/

Scientist, N. (n.d.). How to sniff out the good coronavirus studies from the bad. New Scientist. Retrieved July 1, 2020, from https://www.newscientist.com/article/2242835-how-to-sniff-out-the-good-coronavirus-studies-from-the-bad/

aaronpeikert. (2020). Aaronpeikert/reproducible-research [TeX]. https://github.com/aaronpeikert/reproducible-research (Original work published 2019)

Lab to field. (n.d.). Behaviourally Informed Organizations. Retrieved June 21, 2020, from https://www.biorgpartnership.com/lab-to-field

Marshall, M. (n.d.). Scotland could eliminate the coronavirus – if it weren’t for England. New Scientist. Retrieved July 2, 2020, from https://www.newscientist.com/article/2247462-scotland-could-eliminate-the-coronavirus-if-it-werent-for-england/

r/BehSciMeta - Comment by u/VictorVenema on ”Can one distinguish between argument and fact? And, if yes, how?”. (n.d.). Reddit. Retrieved July 10, 2020, from https://www.reddit.com/r/BehSciMeta/comments/ho0qr1/can_one_distinguish_between_argument_and_fact_and/fxezalm

r/BehSciMeta—No appeasement of bad faith actors. (n.d.). Reddit. Retrieved June 2, 2020, from https://www.reddit.com/r/BehSciMeta/comments/gv0y99/no_appeasement_of_bad_faith_actors/

Clements, J. C. (2020). Don’t be a prig in peer review. Nature. https://doi.org/10.1038/d41586-020-02512-0

Welcome! You are invited to join a meeting: ASAPbio Community Call August 2020. After registering, you will receive a confirmation email about joining the meeting. (n.d.). Zoom Video. Retrieved September 2, 2020, from https://us02web.zoom.us/meeting/register/tZMtc-ysqDIjGNUKrYAoTHYzYeLu9Jd8iP_V

James Evans: The social limits of scientific certainty (Video). (n.d.). Metascience.com. Retrieved September 1, 2020, from https://metascience.com/events/metascience-2019-symposium/james-evans-social-limits-of-scientific-certainty/

r/BehSciResearch—New research project on managing disagreement. (n.d.). Reddit. Retrieved July 27, 2020, from https://www.reddit.com/r/BehSciResearch/comments/hwjm0w/new_research_project_on_managing_disagreement/

van Smeden, M., Lash, T. L., & Groenwold, R. H. H. (2020). Reflection on modern methods: Five myths about measurement error in epidemiological research. International Journal of Epidemiology, 49(1), 338–347. https://doi.org/10.1093/ije/dyz251

Welcome to a new ERA of reproducible publishing. (2020, August 24). ELife; eLife Sciences Publications Limited. https://elifesciences.org/labs/dc5acbde/welcome-to-a-new-era-of-reproducible-publishing

Adams, R. C., Sumner, P., Vivian-Griffiths, S., Barrington, A., Williams, A., Boivin, J., Chambers, C. D., & Bott, L. (2017). How readers understand causal and correlational expressions used in news headlines. Journal of Experimental Psychology: Applied, 23(1), 1–14. https://doi.org/10.1037/xap0000100

CRediT - Contributor Roles Taxonomy. (n.d.). CASRAI. Retrieved June 13, 2020, from https://casrai.org/credit/

Fell, M. J., Pagel, L., Chen, C., Goldberg, M. H., Herberz, M., Huebner, G., Sareen, S., & Hahnel, U. J. J. (2020). Validity of energy social research during and after COVID-19: Challenges, considerations, and responses [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/pe6cd

Moreau, D., & Gamble, B. (2020). Conducting a Meta-Analysis in the Age of Open Science: Tools, Tips, and Practical Recommendations [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/t5dwg

Independent SAGE on Twitter: “NEW: Independent SAGE has evaluated the scientific evidence on social distancing & concludes it is not safe to reduce it from 2m to 1m indoors as government proposes https://t.co/GHgJ6SXW7C” / Twitter. (n.d.). Twitter. Retrieved June 22, 2020, from https://twitter.com/independentsage/status/1274727763786809344

Nature. (2020, April 17). Coronapod: Troubling news. https://play.acast.com/s/nature/coronapod-troublingnews

Chang, D. C., & Stapleton, S. M. (2020). Response: The Proliferation and Misinterpretation of “As Safe As” Statements in Surgical Science: A Call for Professional Discourse to Search for a Solution. Journal of Surgical Research, 0(0). https://doi.org/10.1016/j.jss.2020.03.074

Althouse, A. D. (2020). Post Hoc Power: Not Empowering, Just Misleading. Journal of Surgical Research, 0(0). https://doi.org/10.1016/j.jss.2019.10.049

Bababekov, Y. J., Hung, Y.-C., Hsu, Y.-T., Udelsman, B. V., Mueller, J. L., Lin, H.-Y., Stapleton, S. M., & Chang, D. C. (2019). Is the Power Threshold of 0.8 Applicable to Surgical Science?—Empowering the Underpowered Study. Journal of Surgical Research, 241, 235–239. https://doi.org/10.1016/j.jss.2019.03.062

Nelson, N. C., Ichikawa, K., Chung, J., & Malik, M. (2020). Mapping the discursive dimensions of the reproducibility crisis: A mixed methods analysis [Preprint]. MetaArXiv. https://doi.org/10.31222/osf.io/sbv3q

Fife, D., Lung, M., Sullivan, N., & Young, C. (2020). When Values Collide: Why Scientists Argue About Open Science and How to Move Forward [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/q9d28

Preprint Servers Have Changed Research Culture in Many Fields. Will a New One for Education Catch On? - EdSurge News. (2020, August 20). EdSurge. https://www.edsurge.com/news/2020-08-20-preprint-servers-have-changed-research-culture-in-many-fields-will-a-new-one-for-education-catch-on

Marcus, A. A. (2020, August 16). Hydroxychloroquine, push-scooters, and COVID-19: A journal gets stung, and swiftly retracts. Retraction Watch. https://retractionwatch.com/2020/08/16/hydroxychloroquine-push-scooters-and-covid-19-a-journal-gets-stung-and-swiftly-retracts/

Moshontz, Hannah, Grace Elizabeth Binion, Benjamin T. Brown, and Haley Walton. ‘A Guide to Self-Archiving Preprints’. Preprint. PsyArXiv, 21 August 2020. https://doi.org/10.31234/osf.io/dp4x9.

Ophir, Yaakov, and Yaffa Shir Raz. ‘Manipulations and Spins in Attention Disorders Research: The Case of ADHD and COVID-19’. Preprint. PsyArXiv, 20 August 2020. https://doi.org/10.31234/osf.io/dmu4j.

Experiments have shown that simply visualizing an can have great impacts.

Mather, N. (2020). How we accelerated clinical trials in the age of coronavirus. Nature, 584(7821), 326–326. https://doi.org/10.1038/d41586-020-02416-z

Schalkwyk, M. C. I. van, Hird, T. R., Maani, N., Petticrew, M., & Gilmore, A. B. (2020). The perils of preprints. BMJ, 370. https://doi.org/10.1136/bmj.m3111. https://t.co/qNPLYCeT99?amp=1

Andrew Althouse on Twitter: “@brnichols8744 @JeremySussman @FinancialGonzo @venkmurthy Many scientists use Twitter to carry on conversations (with varying degrees of formality) about published papers, the good, bad, and ugly. The people in this conversation all do this frequently. None of us are anti-science (cont...)” / Twitter. (n.d.). Twitter. Retrieved August 18, 2020, from https://twitter.com/ADAlthousePhD/status/1295168734219337738

Laghaie, A., & Otter, T. (2020). Measuring evidence for mediation in the presence of measurement error [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/5bz3f

Besançon, L., Peiffer-Smadja, N., Segalas, C., Jiang, H., Masuzzo, P., Smout, C., Deforet, M., & Leyrat, C. (2020). Open Science Saves Lives: Lessons from the COVID-19 Pandemic. BioRxiv, 2020.08.13.249847. https://doi.org/10.1101/2020.08.13.249847

C. L., & Print. (2020, August 14). Op-Ed: We rely on science. Why is it letting us down when we need it most? Los Angeles Times. https://www.latimes.com/opinion/story/2020-08-14/replication-crisis-science-cancer-memory-rewriting

Paris, Marseille named as high-risk COVID zones, making curbs likelier. (2020, August 14). Reuters. https://uk.reuters.com/article/uk-health-coronavirus-france-idUKKCN25A0LC

Speelman, C., & McGann, M. (2020). Statements about the Pervasiveness of Behaviour Require Data about the Pervasiveness of Behaviour [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/bxzm4

Michael Eisen on Twitter: “A core problem in science publishing today is that we have a system where the complex, multidimensional assessment of the rigor, validity, utility, audience and impact of a work that emerges from peer review gets reduced to a single overvalued ‘accept/reject’ decision.” / Twitter. (n.d.). Twitter. Retrieved August 10, 2020, from https://twitter.com/mbeisen/status/1291752487448276992

Gewin, V. (2020). The trials of global research under the coronavirus. Nature. https://doi.org/10.1038/d41586-020-02326-0

Collins, G. S., & Wilkinson, J. (n.d.). Statistical issues in the development a COVID-19 prediction models. Journal of Medical Virology, n/a(n/a). https://doi.org/10.1002/jmv.26390

Parikh, D. (2020, May 27). How we write rebuttals. Medium. https://medium.com/@deviparikh/how-we-write-rebuttals-dc84742fece1

Take away: In their 2019 report the WHO actually recommended for, not against, the use of masks in severe influenza epidemics or pandemics, contrasting the statement made in this tweet. Further, recent evidence overwhelmingly supports the benefit of masks for preventing the spread of SARS-CoV2, the virus that causes COVID-19.

The claim: Overall the claim here appears to be that masks are ineffective against the spread of SARS-CoV2, the virus that causes the clinical syndrome known as COVID-19. The evidence used in support of this claim is that “the WHO found ‘moderate’ evidence AGAINST using masks” in their 2019 report on the use of non-pharmaceutical interventions for mitigating influenza pandemics.

The evidence: This overall claim is poorly supported by data and the evidence used to support this claim is incorrectly characterized by the claimant. Narrowly, the claim that the WHO recommended against mask use is patently false. In their report, the WHO reviewed 10 separate studies and did conclude that there was scant evidence that masks significantly decreased spread of the flu. However, they found no evidence that masks increased spread, and based on mechanistic plausibility (i.e. masks are barriers that prevent droplets from passing between people) and the low risk/high reward, they made a conditional recommendation for mask use in severe influenza epidemics or pandemics.

While influenza does not behave exactly like the SARS-CoV2 virus, the similarities in mode of transmission make it reasonably likely that masks would also have protective effects against the spread of this virus is well. The best evidence is hard data, and that too increasingly points to the benefit of masks for slowing down or preventing the transmission of SARS-CoV2. A recent summary of that data is available here.

Fry, C. V., Cai, X., Zhang, Y., & Wagner, C. S. (2020). Consolidation in a crisis: Patterns of international collaboration in early COVID-19 research. PLOS ONE, 15(7), e0236307. https://doi.org/10.1371/journal.pone.0236307

Taraborelli, D., Taraborelli, D., & Taraborelli, D. (2020, August 5). How the COVID-19 crisis has prompted a revolution in scientific publishing. Fast Company. https://www.fastcompany.com/90537072/how-the-covid-19-crisis-has-prompted-a-revolution-in-scientific-publishing

Sample, I. (2020). Secrecy has harmed UK government's response to Covid-19 crisis, says top scientist. Retrieved 4 August 2020, from https://www.theguardian.com/world/2020/aug/02/secrecy-has-harmed-uk-governments-response-to-covid-19-crisis-says-top-scientist

van Rooij, I., & Baggio, G. (2020). Theory development requires an epistemological sea change [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/rnw9q

Travis Whitfill MPH on Twitter: “A quick visual aid of major studies & levels of evidence against #hydroxychloroquine for the use in COVID-19 patients. No robust studies have found any type of benefit for HCQ. https://t.co/YbSjvaoEoO” / Twitter. (n.d.). Twitter. Retrieved August 2, 2020, from https://twitter.com/twhitfill/status/1288825416975708161

adams, jimi, & Light, R. (2020). What Role Does Collaboration have in Responding to COVID-19? [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/jqwyr

Martin, G., Hanna, E., & Dingwall, R. (2020). Face masks for the public during Covid-19: An appeal for caution in policy [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/uyzxe

Mikolai, J., Keenan, K., & Kulu, H. (2020). Household level health and socio-economic vulnerabilities and the COVID-19 crisis: An analysis from the UK [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/4wtz8