How relevant are the systematic reviews in the cochrane library to emergency medical practice?

doi:10.1067/mem.2002.120795

Annals of Emergency Medicine

Volume 39, Issue 2, February 2002, Pages 153-158

https://doi.org/10.1067/mem.2002.120795 Get rights and content

Abstract

Study Objectives: The Cochrane Database of Systematic Reviews (CDSR), one of the databases in the Cochrane Library, is a compilation of regularly updated systematic reviews with meta-analytic summary statistics. We conducted a study of the CDSR to evaluate the relevance of this database to emergency medical practice. Methods: Using criteria that were determined a priori, 2 reviewers independently screened the titles of 795 completed reviews from the April 2000 CDSR for possible relevance to emergency medicine practice. Five reviewers independently ranked summaries of these reviews for degree of relevance. Agreement was measured using κ statistics. Disagreements were resolved by consensus or adjudication. Results: Screening of 795 completed reviews identified 136 (17%) as “possibly” and 151 (19%) as “likely” relevant (simple agreement, 74%; κ=0.82). Further independent assessment indicated 95 (12%) of the 151 “likely” systematic reviews were directly relevant to emergency medicine practice, whereas 44 (6%) were indirectly relevant (simple agreement, 77%; κ=0.45). Cochrane Review Groups producing the most emergency medicine–relevant systematic reviews included acute respiratory infections (14 [10%]), injuries (14 [10%]), pregnancy and childbirth (13 [10%]), stroke (12 [10%]), and airways (11 [8%]). In contrast, only 1 (0.1%) of the reviews produced by the Heart Review Group was considered directly relevant to emergency medicine practice. Conclusion: More than one third of CDSR reviews have some relevance, and 12% are directly relevant, to emergency medical practice or discharge planning. Although certain conditions are well covered, other key emergency medicine areas are not. Emergency physicians should consider the Cochrane Library an important evidence-based resource for emergency medicine therapeutic interventions and should examine ways of increasing the number of reviews relevant to emergency medicine. [Emond SD, Wyer PC, Brown MD, Cordell WH, Spooner CH, Rowe BH. How relevant are the systematic reviews in the Cochrane Library to emergency medical practice? Ann Emerg Med. February 2002;39:153-158.]

Introduction

Advances in biomedical knowledge continue to expand the information available to guide clinicians in the care of patients. To effectively translate this evidence into practice, a clinician needs to access relevant data, determine its validity, and then decide whether it applies to a given patient. These tasks are particularly challenging in the emergency setting, where the scope of practice crosses numerous medical disciplines and the pace of decisionmaking can be frenetic.

In this context, traditional resources such as textbooks and MEDLINE searches are of limited value in identifying the best evidence to guide clinical practice. However, the promise of evidence-based medicine resources is to provide the busy clinician with rapidly accessible synthesized information using valid and reliable approaches. The Cochrane Library, a compendium of databases and related instructional tools, has been promoted as just such a reference.¹

The Cochrane Library is composed of several databases, including the Cochrane Controlled Trials Register (CCTR), the database of Abstracts of Reviews of Effectives (DARE), and the Cochrane Database of Systematic Reviews (CDSR). The CDSR database is a compilation of regularly updated systematic reviews with meta-analytic summary statistics. Collaborative review groups representing various medical topic areas (eg, airways, musculoskeletal, heart, epilepsy) contribute content to the CDSR.

The CDSR consists of protocols and completed reviews. Protocols describe the objectives of systematic reviews that are in the process of being completed. Completed reviews include the full text and usually present summary statistics. The protocols and completed reviews are produced using rigorous methodological standards and undergo peer review before publication. Regular updates are required to capture new evidence and address criticisms and/or identified errors.

Although the CDSR database contains rigorously produced systematic reviews, the relevance of these reviews to emergency medicine practice has not been previously evaluated. We therefore conducted a study to assess the relevance of the CDSR database reviews to emergency medicine practice.

Section snippets

Materials and methods

The study methodology is summarized in Figure 1.

. Study methodology.

For the purposes of this study, we defined “relevant” as “important or essential to emergency medicine patient care or disposition.” Using criteria that were determined a priori, 2 reviewers (SDE, BHR) independently screened the titles of the 795 completed reviews listed in the April 2000, Issue 2, CDSR to identify potentially relevant reviews for further study. Titles were scored using a 3-category scale (Figure 2).

. Screening

Results

Screening of the 795 completed reviews in the CDSR database identified 136 (17%) as “potentially” relevant and 151 (19%) as “likely” relevant (simple agreement, 74%; κ=0.82). Independent scoring of the 151 “likely” relevant reviews indicated that 95 (12%) were “directly” relevant to emergency medical practice and 44 (6%) were “indirectly” relevant to emergency medical practice (simple agreement, 77%; κ=0.45). The remaining reviews were deemed “not relevant” to emergency medical practice.

The

Discussion

The Cochrane Library is produced by the Cochrane Collaboration, an international, multidisciplinary, volunteer network of health care professionals, consumers, researchers, and policymakers and published quarterly by Update Software Ltd.(Oxford, England).1, 2 The goal of the Collaboration is to produce, maintain, and disseminate evidence on therapy. In general, the Cochrane Collaboration uses groupings known as “fields” and “networks” to define dimensions of health care other than health

Acknowledgements

Author contributions: SDE and BHR conceived and designed the study, managed the data, and provided quality control. SDE, PCW, MDB, WHC, and BHR reviewed abstracted data. BHR and CHS provided statistical advice on study design and analyzed the data. SDE drafted the manuscript, and all authors contributed substantially to its revision. SDE assumes responsibility for the paper as a whole.

References (10)

BH Rowe et al.
The Cochrane Library: a resource for clinical problem solving in emergency medicine
Ann Emerg Med
(1999)
AR Jadad et al.
Ten challenges at the intersection of clinical research, evidence-based medicine, and pain relief
Ann Emerg Med
(2000)
CD. Naylor
Grey zones of clinical practice: some limits to evidence-based medicine
Lancet
(1995)
RS Hockberger et al.
The model of the clinical practice of emergency medicine
Ann Emerg Med
(2001)
J Barker et al.
Evidence produced in evidence-based medicine needs to be relevant
BMJ
(2000)

There are more references available in the full text version of this article.

Cited by (17)

The Cochrane Library as a Resource for Evidence on Out-of-Hospital Health Care Interventions
2007, Annals of Emergency Medicine
Citation Excerpt :
However, identifying evidence for out-of-hospital interventions in the Cochrane Library can be challenging. Medical Subject Headings (MeSH) are lacking for out-of-hospital interventions, and conducting a comprehensive search with text terms can be laborious because the scope of out-of-hospital practice crosses many medical disciplines,1 requiring the use of a large number of search terms to identify all existing literature. In 2004, the Cochrane Prehospital and Emergency Health Field was registered as an official entity of the Cochrane Collaboration.
Although the Cochrane Library is promoted as a rigorous source of evidence, the relevance of this evidence to the out-of-hospital setting has not been assessed. The objective of this study is to identify existing controlled trials and systematic reviews in the Cochrane Library that have been conducted in or are relevant to out-of-hospital health care. The scope of out-of-hospital care, the years and trends of out-of-hospital research publication, and the journals of publication are also examined.
Using a search strategy developed by the Cochrane Prehospital and Emergency Health Field, 2 reviewers searched issue 4, 2005 of the Cochrane Library to identify reports of controlled trials and systematic reviews on out-of-hospital interventions. Three independent reviewers screened the titles identified by the search strategy and applied predetermined criteria to classify the reported study as out-of-hospital based or not out-of-hospital based. The out-of-hospital-based studies were then categorized as randomized or nonrandomized trials.
Screening of the 19,759 titles retrieved by the search strategy identified 4,016 studies that were potentially out-of-hospital based. Abstract and full-text analysis of the 4,016 studies identified 400 reports of trials and 13 reviews or protocols that were out-of-hospital based. Of the 400 reports of trials identified, 299 (75%) were randomized trials. The number of out-of-hospital trials published increased steadily from the 1970s to the late 1990s, with the majority of trials (63%) covering interventions related to resuscitation and cardiac care. Annals of Emergency Medicine published more out-of-hospital trials than any other journal, followed by Resuscitation and Journal of the American Medical Association.
The Cochrane Library provides a useful resource of health care evidence; however, the relatively small number of out-of-hospital-based systematic reviews and trials does not comprehensively cover the broad scope of out-of-hospital health care.
A Primer on the Cochrane Collaboration, Its New Priorities in Out-of-Hospital and Emergency Health, and the Role of Annals of Emergency Medicine
2007, Annals of Emergency Medicine
How to find evidence when you need it, part 4: Matching clinical questions to appropriate databases
2003, Annals of Emergency Medicine
Citation Excerpt :
The databases of the Cochrane Library pertain to all medical specialities, although to varying degrees. A structured review of the Cochrane Database of Systematic Reviews in April 2000 found that up to 18% of the reviews were at least indirectly relevant to emergency practice.17 The Cochrane Controlled Trials Register currently includes almost 350,000 citations and is the largest such registry in existence.
Practitioners seeking enhancement of clinical care through consideration of research require rapid and efficient point-of-care access to current studies and summaries pertaining to specific clinical queries. MEDLINE and other large databases usually contain the citations relevant to such questions but frequently fall short of the practical requirements of busy clinicians. We present a summary of the knowledge and skills required for physicians to select and use smaller databases appropriate to particular types of questions arising from emergency care. We outline a step-by-step approach that begins at the bedside with the sorting of questions into appropriate categories of knowledge and research design. We identify commonly encountered pitfalls in the process of connecting a particular question to an appropriate database. We illustrate the approach through a set of demonstration questions pertaining to patients presenting to emergency departments with chest pain consistent with acute coronary ischemia. We describe a selection of resources and databases and summarize their performance in locating articles relevant to the demonstration questions.
Improving the dissemination of systematic reviews in emergency medicine
2002, Annals of Emergency Medicine
Effective Synthesized/preappraised Evidence Formats in Emergency Medicine and the Use of Supplemental Knowledge Translation Techniques
2007, Academic Emergency Medicine
Citation Excerpt :
There are a variety of synthesized evidence products available in the medical literature, and several of them pertain to EM.9 The most widely recognized and perhaps highest-quality resource is the Cochrane Library,10,11 and research has shown that this evidence is relevant to EM practice.9 Several HTA resources exist in general medical topics; however, none appear dedicated to the emergency field.
Most clinicians, and especially emergency physicians, are increasingly faced with the need for valid and reliable evidence upon which to base practice decisions in a timely fashion. Despite the accumulation of synthesized evidence in emergency medicine over the past decade, knowledge gaps still exist between what is known and what is practiced. In many cases, this failure in knowledge uptake relates to barriers in uptake as well as the difficulty of translating evidence from research to the bedside. Preappraised evidence syntheses represent a potential partial solution to these problems by providing condensed summaries of the large volume of scientific literature in our field. The participants in this workshop examined the availability, utility, and impact of preappraised evidence and examined innovative ways to translate this knowledge into practice. In addition, the workshop participants also explored more globally all knowledge translation methods that are distinct from clinical pathways (e.g., audit and feedback, academic detailing, reminders, and local opinion leaders). These are initiatives that are instituted at the level of a particular hospital or with respect to a certain condition, and emergency physicians need to understand their definition and application. Overall, the recommendations arising from this workshop have the potential to alter future emergency care in important ways.
Informatics and Knowledge Translation
2007, Academic Emergency Medicine
To ensure that the benefits of knowledge translation synthesis are accessible to care providers at the point of decision-making, fast, efficient, usable clinical information systems are required. Medical informatics appears to hold the greatest promise to be able to create systems with the necessary capacity and functionality. Emergency medicine needs to be actively engaged at all levels of the process. This includes driving the development and filtering of emergency-specific synopses and summaries. It requires advocating for hardware and software that suit the needs of the emergency department environment. It is increasingly important to educate and participate on committees with funders and policy-makers to ensure they support this growing evolution. To determine the outcome of these initiatives, careful evaluation is required to inform the discussion. End-users need to be actively involved in the development and usability testing of clinical information retrieval technology and clinical decision-support systems and make certain relevant best evidence is readily accessible and formatted to meet the needs of the working emergency physician. The integration of knowledge translation into clinical practice, and the impact of delivering electronic clinical decision-support, requires methodologically sound studies to confirm or refute its benefits and guide future development of medical informatics.

View all citing articles on Scopus

^*: Author contributions are provided at the end of this article.

^**: Reprints not available from the authors.

View full text

Original ContributionsHow relevant are the systematic reviews in the cochrane library to emergency medical practice?*,**

Abstract

Introduction

Section snippets

Materials and methods

Results

Discussion

Acknowledgements

Ann Emerg Med

Ann Emerg Med

Lancet

Ann Emerg Med

Evidence produced in evidence-based medicine needs to be relevant

BMJ

Original Contributions
How relevant are the systematic reviews in the cochrane library to emergency medical practice?*,**