Evaluating the information retrieval quality and methodological accuracy of Systematic reviews and Meta-analysis on of congenital malformations (2004-2014).

Abstract

Introduction Systematic reviews (SR) and meta-analysis (MA) aim to provide an in-depth summary of the literature of a research question, which must achieve some methodological requirements especially regarding how the information is retrieved and organized. There are several guidelines with recommendations for standard SRs or MAs. However, how often do those publications fulfil all the conditions to be considered SRs or MAs? Objectives Our aim is to check if articles using the terms 'systematic review' or 'meta-analysis' in the title accomplish the established requirements, focusing on search and methodology. The secondary objective is to observe if librarians have participated in a visible manner in the process. Methods We first created a checklist starting with some PRISMA points related to the literature search methodology and the documentation of the process. We added other common elements from the main methodological manuals for SR (including CRD, Cochrane, EUnetHTA, among others). Finally, we completed it with some items of the CADTH Checklist. Our final list consists of 20 evaluation criteria within the subject ‘congenital malformations’. To obtain the sample we searched in Medline/Pubmed and Embase for documents published between 2004 and 2014 and containing the terms SR or MA in their title. We limited languages to English, French, German, Italian and Spanish. We obtained 162 records after excluding duplicates and non-valid documents (letters, etc.). Once we obtain the full texts, we independently checked if the publications met our criteria. A second reviewer was consulted in cases of doubt. Results Among all the data, we highlight the following: - Around 80% do not show PICO’s questions, and around 60% specify bias - Information sources are explained in approximately 70% of the records. Around 60% describe the fully search strategies and nearly 50% combine electronic with manual searches - 20% of them use other additional sources or other types of documents - Around 30% use a thesaurus, and a similar number combines controlled vocabulary with natural language - Less than 10% of the studies mentioned a librarian Conclusions Although we cannot affirm that our sample is sufficiently representative, the fact remains that since most of the studies analysed are lacking in method and resources, and that is quite alarming. Authors and publishers must bear in mind the existing guidelines. Additionally, the involvement of information specialists would be a key factor in improving the quality of SR and MA.