Meta-analysis reproducibility in speech, language, and hearing research (Chow et al., 2021)

Purpose: The purposes of this meta-review are to (a) articulate the importance of transparency and reproducibility in meta-analysis, (b) assess the transparency and reproducibility of meta-analyses published in journals of the American Speech-Language-Hearing Association (ASHA), and (c) discuss the implications of our findings and recommendations for future research.

Method: We conducted a meta-review of all meta-analyses published in ASHA journals through December 31, 2020. Our systematic review yielded 47 meta-analyses for inclusion in this review. We coded all eligible reports on the core elements of transparency and reproducibility in meta-analysis.

Results: Our findings suggest that though reporting tendencies have improved over time, much work is needed to promote transparency and reproducibility in meta-analytic work. Key areas for future accountability include preregistering study protocol, using Preferred Reporting in Systematic Reviews and Meta-Analysis checklists, providing full data sets, and publishing analytic codes.

Conclusions: The state of reporting in meta-analysis is improving over time. We conclude with a discussion of specific areas that need further attention, and recommendations for researchers to consider when conducting future meta-analyses.

Supplemental Material S1. References of the included studies.

Supplemental Material S2. Summary of included studies.

Supplemental Material S3. Coding manual.

Supplemental Material S4. Decision rules for some complicated coding variables.

Supplemental Material S5. Complete data.

Chow, J. C., Sjogren, A. L., & Zhao, H. (2021). Reporting and reproducibility of meta-analysis in speech, language, and hearing research. Journal of Speech, Language, and Hearing Research. Advance online publication. https://doi.org/10.1044/2021_JSLHR-21-00047