How effective is second language incidental vocabulary learning? A meta-analysis

The units are only "comparable across studies" if there weren't any mishaps (eg, clinical or methodological heterogeneity). If there's clinical heterogeneity, then we're probably comparing apples to oranges (ie, either participants, interventions, or outcomes are different among studies). If there's methodological heterogeneity, then that means there's a difference in study design

PDF summary by Cochrane for planning a meta-analysis at the protocol stage. Gives guidance on how to anticipate & deal with various types of heterogeneity (clinical, methodological , & statistical). Link to paper

Covers - ways to assess heterogeneity - courses of action if substantial heterogeneity is found - methods to examine the influence of effect modifiers (either to explore heterogeneity or because there's good reason to suggest specific features of participants/interventions/study types will influence effects of the intervention. - methods include subgroup analyses & meta-regression

If the statistical heterogeneity is larger that what's expected by chance alone, then what does that imply? That there's either clinical or methodological heterogeneity within the pooled studies.

What's the impact of the presence of clinical heterogeneity? The statistical heterogeneity (variation of effects/results of interventions) becomes greater than what's expected by chance alone

What's happens if methodological heterogeneity is present? The statistical heterogeneity (variation of effects/results of interventions) becomes greater than what's expected by chance alone

Haas, J. W., Bender, F. L., Ballou, S., Kelley, J. M., Wilhelm, M., Miller, F. G., Rief, W., & Kaptchuk, T. J. (2022). Frequency of Adverse Events in the Placebo Arms of COVID-19 Vaccine Trials: A Systematic Review and Meta-analysis. JAMA Network Open, 5(1), e2143955. https://doi.org/10.1001/jamanetworkopen.2021.43955

Nerd, G. M.-K. H. (2022, February 25). The Jury is Still Out on Ivermectin. Medium. https://gidmk.medium.com/the-jury-is-still-out-on-ivermectin-7d0a1895549

Nehal, K. R., Steendam, L. M., Campos Ponce, M., van der Hoeven, M., & Smit, G. S. A. (2021). Worldwide Vaccination Willingness for COVID-19: A Systematic Review and Meta-Analysis. Vaccines, 9(10), 1071. https://doi.org/10.3390/vaccines9101071

Lopez-Leon, S., Wegman-Ostrosky, T., Valle, N. C. A. del, Perelman, C., Sepulveda, R., Rebolledo, P. A., Cuapio, A., & Villapol, S. (2022). Long COVID in Children and Adolescents: A Systematic Review and Meta-analyses. (p. 2022.03.10.22272237). medRxiv. https://doi.org/10.1101/2022.03.10.22272237

Peter R. Hansen. (2022, February 3). Weighting, is the answer. The only study to find lockdowns ⬆️mortality is given weight 91.8% = 7390/8030, and then you get -0.2% to be the estimate. To summarize: -0.2% META-STUDY ESTIMATE is based on 91.8% ONE STUDY and 8.2% ALL OTHER STUDIES. https://t.co/j6e7ziPNAI [Tweet]. @ProfPHansen. https://twitter.com/ProfPHansen/status/1489366528956919808

Sah, P., Fitzpatrick, M. C., Zimmer, C. F., Abdollahi, E., Juden-Kelly, L., Moghadas, S. M., Singer, B. H., & Galvani, A. P. (2021). Asymptomatic SARS-CoV-2 infection: A systematic review and meta-analysis. Proceedings of the National Academy of Sciences, 118(34). https://doi.org/10.1073/pnas.2109229118

replicationnetwork. (2021, May 18). DUAN & REED: How Are Meta-Analyses Different Across Disciplines? The Replication Network. https://replicationnetwork.com/2021/05/18/duan-reed-how-are-meta-analyses-different-across-disciplines/

Okoli, G. N., Abou-Setta, A. M., Neilson, C. J., Chit, A., Thommes, E., & Mahmud, S. M. (2019). Determinants of Seasonal Influenza Vaccine Uptake Among the Elderly in the United States: A Systematic Review and Meta-Analysis. Gerontology and Geriatric Medicine, 5, 2333721419870345. https://doi.org/10.1177/2333721419870345

Jonathan Robinson on Twitter. (n.d.). Twitter. Retrieved 6 September 2021, from https://twitter.com/jon_m_rob/status/1431734411335176199

Rogers, J. P., Watson, C. J., Badenoch, J., Cross, B., Butler, M., Song, J., Hafeez, D., Morrin, H., Rengasamy, E. R., Thomas, L., Ralovska, S., Smakowski, A., Sundaram, R. D., Hunt, C. K., Lim, M. F., Aniwattanapong, D., Singh, V., Hussain, Z., Chakraborty, S., … Rooney, A. G. (2021). Neurology and neuropsychiatry of COVID-19: A systematic review and meta-analysis of the early literature reveals frequent CNS manifestations and key emerging narratives. Journal of Neurology, Neurosurgery & Psychiatry, jnnp-2021-326405. https://doi.org/10.1136/jnnp-2021-326405

Parsons, J. E., Newby, K. V., & French, D. P. (2018). Do interventions containing risk messages increase risk appraisal and the subsequent vaccination intentions and uptake? – A systematic review and meta‐analysis. British Journal of Health Psychology, 23(4), 1084–1106. https://doi.org/10.1111/bjhp.12340

Lytras, T., Kopsachilis, F., Mouratidou, E., Papamichail, D., & Bonovas, S. (2015). Interventions to increase seasonal influenza vaccine coverage in healthcare workers: A systematic review and meta-regression analysis. Human Vaccines & Immunotherapeutics, 12(3), 671–681. https://doi.org/10.1080/21645515.2015.1106656

Racine, Nicole, Rachel Eirich, Jessica Cookee, Jenney Zhu, Paolo Pador, Nicole Dunnewold, and Sheri Madigan. ‘When the Bough Breaks: A Systematic Review and Meta-Analysis of Mental Health Symptoms in Mothers of Young Children during the COVID-19 Pandemic’. PsyArXiv, 7 May 2021. https://doi.org/10.31234/osf.io/u8pnh.

Brewer, N. T., DeFrank, J. T., & Gilkey, M. B. (2016). Anticipated Regret and Health Behavior: A Meta-Analysis. Health Psychology : Official Journal of the Division of Health Psychology, American Psychological Association, 35(11), 1264–1275. https://doi.org/10.1037/hea0000294

Niemeyer, H., Aert, R. C. M. van, Schmid, S., Uelsmann, D., Knaevelsrud, C., & Schulte-Herbrueggen, O. (2020). Publication Bias in Meta-Analyses of Posttraumatic Stress Disorder Interventions. Meta-Psychology, 4. https://doi.org/10.15626/MP.2018.884

Chen, X., Chen, Z., Azman, A. S., Deng, X., Sun, R., Zhao, Z., Zheng, N., Chen, X., Lu, W., Zhuang, T., Yang, J., Viboud, C., Ajelli, M., Leung, D. T., & Yu, H. (2021). Serological evidence of human infection with SARS-CoV-2: A systematic review and meta-analysis. The Lancet Global Health, 0(0). https://doi.org/10.1016/S2214-109X(21)00026-7

Ioannidis, John P. A. (2020) ‘The Infection Fatality Rate of COVID-19 Inferred from Seroprevalence Data’. MedRxiv. https://doi.org/10.1101/2020.05.13.20101253.

Peiffer-Smadja, Nathan, Mathieu E. Rebeaud, Anthony Guihur, Yahya Mahamat-Saleh, and Thibault Fiolet. ‘Hydroxychloroquine and COVID-19: A Tale of Populism and Obscurantism’. The Lancet Infectious Diseases 0, no. 0 (13 November 2020). https://doi.org/10.1016/S1473-3099(20)30866-5.

Gligorić, Vukašin, Allard Feddes, and Bertjan Doosje. ‘Political Bullshit Receptivity and Its Correlates: A Cross-Cultural Validation of the Concept’. PsyArXiv, 27 October 2020. https://doi.org/10.31234/osf.io/u9pe3.

ReconfigBehSci. (2020, October 27). RT @JASPStats: How to perform Robust Bayesian Meta-Analysis in JASP. To learn more, have a look at the tutorial video: Https://t.co/4fmkLEH… [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1321387314887708672

Levin, A. T., Hanage, W. P., Owusu-Boaitey, N., Cochran, K. B., Walsh, S. P., & Meyerowitz-Katz, G. (2020). Assessing the age specificity of infection fatality rates for COVID-19: Systematic review, meta-analysis, and public policy implications. European Journal of Epidemiology, 35(12), 1123–1138. https://doi.org/10.1007/s10654-020-00698-1

McCarrick, D., Prestwich, A., Prudenzi, A., & O’Connor, D. (2021). Health Effects of Psychological Interventions for Worry and Rumination: A Meta-analysis. PsyArXiv. https://doi.org/10.31234/osf.io/bsf9e

Tomfohr-Madsen, L., Giesbrecht, G., Lebel, C., Racine, N., & Madigan, S. (2021). Depression and Anxiety in Pregnancy during COVID-19: A Rapid Review and Meta-Analysis. PsyArXiv. https://doi.org/10.31234/osf.io/n8b7x

Spadaro, G., Tiddi, I., Columbus, S., Jin, S., Teije, A. t., & Balliet, D. (2020, October 28). The Cooperation Databank. https://doi.org/10.31234/osf.io/rveh3

Brewer NT, Chapman GB, Gibbons FX, Gerrard M, McCaul KD, Weinstein ND. Meta-analysis of the relationship between risk perception and health behavior: the example of vaccination. Health Psychol. 2007;26(2):136–45. doi:10.1037/0278-6133.26.2.136

Weingarten. E., Chen. Q., McAdams., Yi. J., (2016). From Primed Concepts to Action: A Meta-Analysis of the BehavioralEffects of Incidentally Presented Words. Psychological Bulletin 2016 (142) pp 472-497.

Fischer, R., & Karl, J. (2020). Predicting behavioral intentions to prevent or mitigate COVID-19: A meta-analysis. PsyArXiv. https://doi.org/10.31234/osf.io/ek69g

Offeddu, V., Yung, C. F., Low, M. S. F., & Tam, C. C. (2017). Effectiveness of Masks and Respirators Against Respiratory Infections in Healthcare Workers: A Systematic Review and Meta-Analysis. Clinical Infectious Diseases, 65(11), 1934–1942. https://doi.org/10.1093/cid/cix681

Moreau, D., & Gamble, B. (2020). Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations. Psychological Methods, No Pagination Specified-No Pagination Specified. https://doi.org/10.1037/met0000351

Siemieniuk, R. A., Bartoszko, J. J., Ge, L., Zeraatkar, D., Izcovich, A., Kum, E., Pardo-Hernandez, H., Rochwerg, B., Lamontagne, F., Han, M. A., Liu, Q., Agarwal, A., Agoritsas, T., Chu, D. K., Couban, R., Darzi, A., Devji, T., Fang, B., Fang, C., … Brignardello-Petersen, R. (2020). Drug treatments for covid-19: Living systematic review and network meta-analysis. BMJ, 370. https://doi.org/10.1136/bmj.m2980

Smith, C. F., Drew, S., Ziebland, S., & Nicholson, B. D. (2020). Understanding the role of GPs’ gut feelings in diagnosing cancer in primary care: A systematic review and meta-analysis of existing evidence. British Journal of General Practice, 70(698), e612–e621. https://doi.org/10.3399/bjgp20X712301

Hohn, R. E., Slaney, K. L., & Tafreshi, D. (2020). An Empirical Review of Research and Reporting Practices in Psychological Meta-Analyses. Review of General Psychology, 1089268020918844. https://doi.org/10.1177/1089268020918844

Moreau, D., & Gamble, B. (2020). Conducting a Meta-Analysis in the Age of Open Science: Tools, Tips, and Practical Recommendations [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/t5dwg

Mohammadi, A., Esmaeilzadeh, E., Li, Y., Bosch, R. J., & Li, J. Z. (2020). SARS-CoV-2 detection in different respiratory sites: A systematic review and meta-analysis. EBioMedicine, 0(0). https://doi.org/10.1016/j.ebiom.2020.102903

Cevik, M., Tate, M., Lloyd, O., Maraolo, A. E., Schafers, J., & Ho, A. (2020). SARS-CoV-2, SARS-CoV-1 and MERS-CoV viral load dynamics, duration of viral shedding and infectiousness: A living systematic review and meta-analysis. MedRxiv, 2020.07.25.20162107. https://doi.org/10.1101/2020.07.25.20162107

Levin, A. T., Cochran, K. B., & Walsh, S. P. (2020). Assessing the Age Specificity of Infection Fatality Rates for COVID-19: Meta-Analysis & Public Policy Implications (Working Paper No. 27597; Working Paper Series). National Bureau of Economic Research. https://doi.org/10.3386/w27597

Walter, N., Brooks, J. J., Saucier, C. J., & Suresh, S. (2020). Evaluating the Impact of Attempts to Correct Health Misinformation on Social Media: A Meta-Analysis. Health Communication, 0(0), 1–9. https://doi.org/10.1080/10410236.2020.1794553

Su, Q., Hu, J., Lin, H., Zhang, Z., Zhu, E. C., Zhang, C., Wang, D., Gao, Z., & Cao, B. (2020). Prevalence and risks of severe events for cancer patients with COVID-19 infection: A systematic review and meta-analysis. MedRxiv, 2020.06.23.20136200. https://doi.org/10.1101/2020.06.23.20136200

Olsson-Collentine, A., van Assen, M. A. L. M., & Wicherts, J. M. (2020). Postprint—Heterogeneity in direct replications in psychology and its association with effect size [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/m23v4

Du, H., Jiang, G., & Ke, Z. (2020). A Bootstrap Based Between-Study Heterogeneity Test in Meta-Analysis [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/de4g9

Edgcumbe, D. (2020). PrePrint Version (Edgcumbe, 2020): The developmental trajectory of open-mindedness: from 18 to 87-years of age. [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/fnrmv

Cheung, M. W.-L. (2020). Meta-Analytic Structural Equation Modeling [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/epsqt

Maier, M., Bartoš, F., & Wagenmakers, E.-J. (2020). Robust Bayesian Meta-Analysis: Addressing Publication Bias with Model-Averaging [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/u4cns

Atoui, S., Chevance, G., Romain, A. J., Kingsbury, C., Lachance, J., & Bernard, P. (2020, June 17). Daily associations between sleep and physical activity: A systematic review and meta-analysis. https://doi.org/10.31234/osf.io/ezusb

Chu, D. K., Akl, E. A., Duda, S., Solo, K., Yaacoub, S., Schünemann, H. J., Chu, D. K., Akl, E. A., El-harakeh, A., Bognanni, A., Lotfi, T., Loeb, M., Hajizadeh, A., Bak, A., Izcovich, A., Cuello-Garcia, C. A., Chen, C., Harris, D. J., Borowiack, E., … Schünemann, H. J. (2020). Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: A systematic review and meta-analysis. The Lancet, 0(0). https://doi.org/10.1016/S0140-6736(20)31142-9

Chu, D., Akl, E., El-Harakeh, A., Bognanni, A., Lotf, T., Loeb, M., ... & Chen, C. (2020). Physical Distancing, Face Masks, and Eye Protection to Prevent Person-Person COVID-19 Transmission: A Systematic Review and Meta-Analysis.

Hanel, P. H. P. (2020). Conducting High Impact Research With Limited Financial Resources (While Working From Home) [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/s3fcu

Buitrago-Garcia, D. C., Egli-Gany, D., Counotte, M. J., Hossmann, S., Imeri, H., Salanti, G., & Low, N. (2020). The role of asymptomatic SARS-CoV-2 infections: Rapid living systematic review and meta-analysis [Preprint]. Epidemiology. https://doi.org/10.1101/2020.04.25.20079103

This is what I'd expect for tobacco, but it tells me little about nicotine. Most of the subjective effects are not from tobacco, so It's still plausible that nicotine does not develop tolerance. Indeed, the effects that don't go away are the effects expected from nicotine.

Moyers, S. A., & Hagger, M. S. (2020, April 20). Physical activity and sense of coherence: A meta-analysis. https://doi.org/10.31234/osf.io/d9e3k

Fischer, R., Karl, J. A., Bortolini, T., Zilberberg, M., Robinson, K., Rabelo, A. L. A., … Mattos, P. (2020, April 22). Rapid review and meta-meta-analysis of self-guided interventions to address anxiety, depression and stress during COVID-19 social distancing. https://doi.org/10.31234/osf.io/ndyf4

Jefferson, T., Jones, M., Al Ansari, L. A., Bawazeer, G., Beller, E., Clark, J., Conly, J., Del Mar, C., Dooley, E., Ferroni, E., Glasziou, P., Hoffman, T., Thorning, S., & Van Driel, M. (2020). Physical interventions to interrupt or reduce the spread of respiratory viruses. Part 1 - Face masks, eye protection and person distancing: Systematic review and meta-analysis [Preprint]. Public and Global Health. https://doi.org/10.1101/2020.03.30.20047217

For a blog post commenting on an earlier draft of this article (many points raised remain relevant) see here.

That part never worked. But maybe these annotations will? That’d be neat.