Improving peer review of systematic reviews by involving librarians and information specialists: protocol for a randomized controlled trial

Authors

Melissa L. Rethlefsen, University of New Mexico Health Sciences Library and Informatics Center, Albuquerque, New Mexico, USAFollow
Sara Schroter, MJ Publishing Group, London, England
Lex M. Bouter, Department of Philosophy, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands; Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
David Moher, Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, Ottawa, Ontario, Canada
Ana Patricia Ayala, Gerstein Science Information Centre, University of Toronto, Toronto, Canada
Jamie J. Kirkham, Centre for Biostatistics, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
Maurice P. Zeegers, Department of Epidemiology, School for Nutrition and Translational Research in Metabolisms, Care and Health Research Institute, MD, Maastricht, The Netherlands

Publication Date

11-11-2021

Abstract

BACKGROUND: Problems continue to exist with the reporting quality and risk of bias in search methods and strategies in systematic reviews and related review types. Peer reviewers who are not familiar with what is required to transparently and fully report a search may not be prepared to review the search components of systematic reviews, nor may they know what is likely to introduce bias into a search. Librarians and information specialists, who have expertise in searching, may offer specialized knowledge that would help improve systematic review search reporting and lessen risk of bias, but they are underutilized as methodological peer reviewers.

METHODS: This study will evaluate the effect of adding librarians and information specialists as methodological peer reviewers on the quality of search reporting and risk of bias in systematic review searches. The study will be a pragmatic randomized controlled trial using 150 systematic review manuscripts submitted to BMJ and BMJ Open as the unit of randomization. Manuscripts that report on completed systematic reviews and related review types and have been sent for peer review are eligible. For each manuscript randomized to the intervention, a librarian/information specialist will be invited as an additional peer reviewer using standard practices for each journal. First revision manuscripts will be assessed in duplicate for reporting quality and risk of bias, using adherence to 4 items from PRISMA-S and assessors' judgements on 4 signaling questions from ROBIS Domain 2, respectively. Identifying information from the manuscripts will be removed prior to assessment.

DISCUSSION: The primary outcomes for this study are quality of reporting as indicated by differences in the proportion of adequately reported searches in first revision manuscripts between intervention and control groups and risk of bias as indicated by differences in the proportions of first revision manuscripts with high, low, and unclear bias. If the intervention demonstrates an effect on search reporting or bias, this may indicate a need for journal editors to work with librarians and information specialists as methodological peer reviewers.

TRIAL REGISTRATION: Open Science Framework. Registered on June 17, 2021, at https://doi.org/10.17605/OSF.IO/W4CK2 .

ISSN

1745-6215

Document Type

Article

Keywords

Humans, Information Services, Librarians, Peer Review, Randomized Controlled Trials as Topic, Research Report

Recommended Citation

Rethlefsen ML, Schroter S, Bouter LM, Moher D, Ayala AP, Kirkham JJ, Zeegers MP. Improving peer review of systematic reviews by involving librarians and information specialists: protocol for a randomized controlled trial. Trials. 2021 Nov 11;22(1):791. doi: 10.1186/s13063-021-05738-z. PMID: 34763714; PMCID: PMC8582191.

Publication Title

Trials