Article Text
Abstract
Introduction Procedural sedation and analgesia (PSA) is commonplace in the ED. Previous studies have identified capnography as a reliable indicator of PSA-induced respiratory depression. This review investigates the potential effect on patient safety of the use of capnography in addition to standard monitoring for adult patients undergoing PSA in the ED.
Methods MEDLINE, Embase, Scopus, CINAHL and Google Scholar were searched systematically for ED studies using capnography during PSA. Data extraction was performed by two independent authors. Using MedCalc V.13.3.3 and Meta-DiSc V.1.4, data were aggregated under the random-effects model and heterogeneity was assessed using Cochran's Q-test and the I2 statistic.
Results Of the 737 studies that were screened, 7 studies met the eligibility criteria, representing a total of 662 patients. The aggregate diagnostic accuracy for capnography identifying an adverse event included a diagnostic OR of approximately 6 (OR: 5.87; 95% CI 2.41 to 14.3; p<0.001), sensitivity 0.82 (95% CI 0.76 to 0.87), specificity 0.6 (95% CI 0.55 to 0.64), negative likelihood ratio 0.3 (95% CI 0.12 to 0.75) and positive likelihood ratio 1.89 (95% CI 1.53 to 2.34). There was a lack of statistical evidence for a difference in the proportion of adverse events detected when capnography was used in addition to standard monitoring (48.8% (95% CI 32.85 to 64.92)) compared with chance alone (50%).
Conclusions There is no firm evidence that capnography provides additional safety compared with standard monitoring alone during PSA in adults in the ED. There is a paucity of published research involving preoxygenated patients who remain on high-flow oxygen throughout PSA. Well-powered randomised controlled trials, employing an accepted adverse event reporting tool in such patients, are required. Until then, we advocate continued compliance with current professional recommendations for the use of capnography during PSA in adults in the ED.
- airway
- analgesia/pain control
- clinical assessment
- emergency department
Statistics from Altmetric.com
Introduction
Procedural sedation and analgesia (PSA) is commonplace in the ED. PSA involves administering sedative medications with or without analgesics to induce a depressed level of consciousness, enabling clinicians to perform procedures effectively while providing pain relief and allowing the patient to maintain airway control independently.1 Patients should be monitored closely for adverse effects.2 Standard monitoring includes pulse rate, ECG, BP, oxygen saturation and RR. Capnography, the non-invasive measurement of the partial pressure of carbon dioxide in an exhaled breath, may be used as an additional parameter of a patient's ventilation to identify adverse events during procedural sedation. This is accomplished by measuring changes in end-tidal carbon dioxide (ETCO2). Previous studies have identified capnography as a useful diagnostic measure of PSA-related adverse events.3 The joint Royal College of Anaesthetists and the Royal College of Emergency Medicine procedural sedation guidelines deem its use mandatory for deep or dissociative sedation and recommended for lighter levels.2 It is also advocated by the American College of Emergency Physicians policy (Level B recommendation).4 However, evidence of its benefit in reducing adverse events and improving patient safety is uncertain.
First, there is no universally agreed definition for a PSA-related adverse event, making it difficult to identify and report adverse events accurately and consistently between studies. Additionally, there are different levels of sedation: deeper levels are associated with an increased rate of adverse effects;5 this needs to be accounted for when comparing studies. Moreover, there is inconsistent oxygen delivery before and during procedures, making interpretation of study findings difficult. Some studies have found that changes in ETCO2 are not related to adverse outcome in PSA,6 ,7 whereas others suggest that capnography is able to identify the onset of adverse events ahead of changes in standard monitoring.8 ,9
This review investigates the potential effect on patient safety of the use of capnography in addition to standard monitoring for adult patients undergoing PSA in the ED. The review focuses on separate markers of patient safety: first, the diagnostic accuracy of capnography alone in detecting PSA-related adverse events and second, the ability of capnography to detect such events before standard monitoring. Finally, the review aims to evaluate the physician interventions based on capnography data.
Materials and methods
Reporting of this systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.10
Search strategy
An electronic search of MEDLINE and Embase via Ovid; Scopus; Cumulative Index to Nursing and Allied Health Literature (CINAHL) via EBSCO; and Google Scholar was conducted. These databases were searched from their inception to 26 July 2015. ‘Capnography’ was included as a search term for publications dating from 1997, when this particular term was first recognised as a medical subject heading (MeSH) term; synonymous search terms were included to capture pre-1997 publications11 (see online supplementary appendix 1 for search strategies). Database searching was supplemented with identification of references from relevant papers; hand-searching of journals; identification of relevant conference proceedings and searching of clinical trial registries. No restrictions, including language or publication type, were applied.
Supplementary appendices
Study selection
Papers were initially screened and excluded on the basis of the relevance of their titles and abstracts. All randomised control trials (RCTs), quasi-randomised control trials (qRCTs) and observational (including cohort) studies that included an analysis of capnography during PSA were included. Published systematic reviews were analysed for their potential to be extended or revised but were excluded from the review and meta-analysis. All other study types were excluded. All potentially relevant studies were retrieved as full manuscripts. Two independent reviewers (CD and RB) applied predefined inclusion and exclusion criteria (table 1) to remove ineligible or duplicate studies. Disagreements were resolved through arbitration by a third independent reviewer (AG).
Data extraction
Data extraction was performed using a data collection form published by the Cochrane Collaboration.12 No eligible study required language translation.
Quality assessment
To assess bias, the Cochrane risk of bias tool13 for RCTs was used alongside an adapted version14 to accommodate studies that were non-randomised with respect to capnography (see online supplementary appendix 2). Additional assessment of methodological quality was carried out using a validated checklist developed by Downs and Black for RCTs and observational studies.15 Studies were labelled as ‘high quality’ (score 25–28), ‘moderate quality’ (score 20–24) and ‘low quality’ (score<20). The overall quality of the evidence was collectively judged using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system.16
Statistical analysis
Statistical analysis was performed using MedCalc V.13.3.3 (MedCalc Software, Ostend, Belgium) unless stated otherwise.
In the meta-analysis evaluating diagnostic accuracy, ORs were calculated as OR=ad/bc, where a is the number of true positives (adverse events detected as such by ETCO2), b the number of false positives (events misclassified as adverse events by ETCO2), c the number of false negatives (adverse events undetected by ETCO2) and d the number of true negatives (absence of adverse events detected as such by ETCO2).3 Fixed-effects (Mantel–Haenszel method)17 and random-effects (Der Simonian–Laird method)18 models were used to estimate aggregate ORs. Using these tests, the significance of the aggregate ORs was assessed in terms of the null hypothesis that OR=1, using the z-test.
In assessing the potential usefulness of capnography in detecting adverse events for patients undergoing PSA, we assumed the recommended requirement that the diagnostic OR should be ‘well above 20’.19
Further evaluation of the diagnostic utility of capnography was carried out by calculating the aggregate positive and negative likelihood ratios using the random effects model. The sensitivity, specificity, positive likelihood ratio (LR) and negative LR were determined for each included study and in aggregate form using the test accuracy software Meta-DiSc (V.1.4).20
Using a binomial model, a meta-analysis delineating the proportion of adverse events identified by changes in capnography compared with changes in standard monitoring was calculated, with 0.5 (50%) taken to represent the threshold proportion for statistical evidence of an improvement in patient safety. The rationale for this was that a proportion of 0.5 for each of the two groups would be achieved by chance alone. The level of clinical significance corresponding to the aggregate proportion was further classified as: 0–50%, non-significant; 51–70% low; 71–85%, moderate; and 86–100%, high. The Freeman–Tukey transformation (arcsine square root transformation)21 was used to calculate the weighted summary proportion under the fixed-effects and random-effects models.
The random-effects model was chosen to accommodate variability between studies in terms of their design, interventions and characterisation of adverse events. Heterogeneity was assessed using Cochran's Q-test and the I2 statistic,22 which represent the percentage of total variation in effects size attributable to between-study heterogeneity rather than within-study heterogeneity due to sampling error.23 I2 values of 25%, 50% and 75% corresponded to low, medium and high percentages of between-study heterogeneity, respectively.13 Statistical significance was set at p<0.05.
Publication bias
Publication bias was minimised with comprehensive literature searching and the inclusion of smaller negative studies. It was planned that funnel plots would be used to detect publication bias if 10 or more eligible studies reported on a particular outcome.24
Results
Study selection
Electronic searching revealed 737 citations. Thirty-nine full-text articles were in turn assessed for eligibility after abstract and title screening and duplicate removal. Thirty-two articles were excluded (see online supplementary appendix 3): eight included less than 50 patients (with seven of these studies meeting additional exclusion criteria), nine did not include an outcome measure related to ETCO2, four did not record ETCO2, four were reviews, three did not analyse ETCO2 data separate to standard monitoring data and three were performed outside the ED. Finally, one potentially relevant study8 was excluded on the basis that adults and children were not considered separately, and attempts at sourcing further information directly from the author were unsuccessful.
The four excluded reviews were not suitable for extension or revision for the current review (three did not evaluate the use of ETCO2 and one was not limited to the ED). Seven studies, representing a total of 662 patients, satisfied the review inclusion criteria. All studies were performed in North America. No potentially relevant pre-existing systematic reviews were identified by the search strategy. Figure 1 presents the corresponding PRISMA flow diagram.
Study characteristics
Of the seven eligible studies, four were observational studies and three were RCTs. Three included capnography as a primary intervention (ie, they analysed the effect of capnography on detecting adverse events as one of their primary aims)6 ,25 ,26 but only one of these was a RCT. This RCT randomised patients to unblinded capnography (‘intervention’) or blinded capnography (‘control’); in the control group treating physicians were blind to the capnography monitoring screen.25 The primary outcome of this study was the effect of capnography on the incidence of hypoxic events. In the two additional RCTs,27 ,28 relevant analyses were reported as nested cohort studies. The studies varied in terms of use of supplementary oxygen and none of the studies reported preoxygenation of patients. All studies used standard monitoring, which included pulse oximetry, heart rate, RR and BP. The definition of an adverse event varied between studies but typically included hypoxia, respiratory depression, hypotension, bradycardia or hospital admission. Complete definitions of an adverse event are included in the ‘outcomes’ of each study (table 2). The commonest procedures requiring PSA were fracture reduction and abscess incision and drainage. Study characteristics are summarised in table 2 and described in full in online supplementary appendix 3.
Study quality assessment
Among the studies included in the meta-analyses, four had a low risk of bias, two a moderate risk of bias and one a high risk of bias. The latter had a high risk of performance and detection bias, declaring that “capnography should be part of routine practice, and thus it would not be ethical to blind our clinicians”.28 Only three studies, including the RCT, attempted to blind staff to ETCO2 data.25 ,27–29 Of these three, one of the cohort studies terminated early when clinicians were unblinded mid-study.29 However, in the RCT studying capnography as a primary intervention, randomisation was appropriately performed using a computer-generated randomisation list.25 In general, exclusions were pertinent and all patients were accounted for, resulting in low attrition bias. The mean Downs and Black score was 23.2 (range=18–28); studies were labelled as ‘high quality’ (score 25–28, n=3), ‘moderate quality’ (score 20–24, n=2) and ‘low quality’ (score<20, n=2). None of the studies gave any details regarding funding. Using funnel plots to detect publication bias was not feasible owing to the small number of studies. Detailed quality assessment is included in online supplementary appendix 4. Online supplementary appendix 5 summarises the overall study quality according to GRADE guidelines.
Definition and detection of adverse events
The most commonly reported outcomes were hypoxia and respiratory depression. Their definitions were heterogeneous among studies. Hypoxia was defined as SpO2 <93% for >15 s; 25 ,27 ,28 SpO2 <92%6 ,29 or SpO2 <90%.26 ,30 Clinically significant respiratory depression was defined by ‘ETCO2 changes’: six studies included loss of ETCO2 waveform6 ,25–28 ,30 and all seven studies included ETCO2 changes ≥10% or ≥10 mm Hg from baseline.
Adverse events were defined separately by each study as one or more of: hypoxia, respiratory depression, hypotension, bradycardia, arrhythmia, vomiting, increase in supplementary oxygen, prolonged ED stay or admission, increase in supplemental oxygen, airway repositioning, physical or verbal stimulation, or reversal agent administration. ‘Positive capnography’ was defined individually by each study in terms of a prespecified change in the ETCO2 trace (table 2). The diagnostic accuracy of capnography to detect these predefined adverse events was calculated as an OR for each study and included in a meta-analysis (figure 2). Overall, the diagnostic OR for capnography identifying an adverse event was approximately 6 (OR: 5.87; 95% CI 2.41 to 14.3; p<0.001). The width of the CI for the aggregate diagnostic OR of 5.87 was fairly high (2.41–14.30). Also, the proportion of between-study heterogeneity was on the high side of moderate (I2=66.88%; 95% CI 26.18 to 85.14), suggesting a moderate-to-high absolute level of between study heterogeneity (figure 2).
Summary data for individual and aggregate sensitivity, specificity, positive and negative likelihood ratios are detailed in figure 3. As indicated in figure 3, on application of Cochran's Q-test for testing for between study heterogeneity, a highly significant effect was found in each case. Further, the I2 statistic suggested moderate levels of between study heterogeneity for the specificity and positive likelihood ratio, but high levels of between study heterogeneity for the sensitivity and negative likelihood ratio.
Capnography versus standard monitoring in detecting adverse events
The RCT investigating capnography as a primary intervention reported that 25% (17/68) of patients with capnography experienced hypoxia (SpO2 <93% for ≥15 s) compared with 42% (27/64) of those with blinded capnography (17% absolute difference; 95% CI 1.3 to 33; p=0.035).25 Other studies compared the number of episodes of respiratory depression detected by capnography with those detected by standard monitoring. One study found that 61% (27/44) of episodes of respiratory depression were identified by ETCO2 changes before pulse oximetry.30 In a second study by Miner et al 26 all episodes (33/33) of respiratory depression were detected by capnography compared with 33% (11/33) by pulse oximetry. A further study found that while 21/36 episodes of respiratory depression were detected by changes in capnography measurements, only 2/36 preceded those detected by changes in pulse oximetry measurements.6
In the meta-analysis that compared detection of adverse events by capnography versus standard monitoring, 48.8% of adverse events were detected by changes in capnography measurements before there were changes in standard monitoring measurements (95% CI 32.85 to 64.92; 7 studies, 662 participants, figure 4). Conversely, 42.0% of adverse events were detected by changes in standard monitoring measurements before changes in capnography measurements. The CI (32.85 to 64.92) for the difference in proportions (48.8) of detected adverse events was fairly wide, while the proportion of between-study heterogeneity was on the low side of moderate (I2=52.14%; 95% CI 78.73 to 93.78), suggesting a moderate level of between-study heterogeneity (figure 4).
Physician intervention
Three studies evaluated physician intervention in response to changes in ETCO2 prior to standard monitoring.25 ,26 ,28 Interventions included verbal or physical stimulation, airway realignment, use of supplementary oxygen or airway adjuncts, assisted ventilation or intubation. In one study, it was found that in patients requiring assisted ventilation, 82% (9/11) were detected by capnography (either an absent ETCO2 waveform or an ETCO2 >50 mm Hg) compared with 18% (2/11) detected by changes in pulse oximetry measurement.26 Other methods of standard monitoring, such as pulse rate, ECG, BP and RR, were not compared for this outcome. In another study, 16/31 interventions were in patients that had ETCO2 changes without hypoxia.27 In the RCT, it was reported that physicians intervened in 35% of cases (24/68) with capnography monitoring compared with 22% (14/64) of cases without capnography monitoring.25 Table 3 includes a summary of these studies.
Supplementary oxygen
The studies varied in terms of use of supplementary oxygen. None of the studies documented the use of preoxygenation of patients during the procedure. Two studies used supplementary oxygen as routine during the procedure,25 ,29 and all other studies used supplementary oxygen as an intervention if a patient developed oxygen desaturation. In the study using high-flow oxygen as an outcome measure, hypoxia was less frequent in patients receiving high-flow oxygen (15 L/min) but there was no statistically significant difference in the capnographic detection of respiratory depression between the two groups.28 The remaining studies did not divide the patients experiencing respiratory depression identified by ETCO2 into those receiving supplementary oxygen and those without supplementary oxygen.
Discussion
This review investigates the potential effect on patient safety of the use of capnography in addition to standard monitoring for adult patients undergoing PSA in the ED. It focuses on the diagnostic accuracy of capnography alone in detecting PSA-related adverse events and the ability of capnography to detect such events before standard monitoring. This review also explores physician interventions based on capnography data.
As a measure of diagnostic accuracy for capnography identifying an adverse event, the statistically significant aggregate diagnostic OR of approximately 6 suggests a relationship between capnography and adverse event detection. This level of diagnostic accuracy is similar to a previous meta-analysis investigating capnography during procedural sedation across a number of settings.3 However, this value and the upper limit of the corresponding 95% CI fall below the lower limit of 20 recognised as corresponding to genuine clinical importance.23
The aggregate sensitivity for detection of an adverse event by capnography was high in comparison to the relative modest corresponding aggregate specificity. These findings, together with the relatively narrow CIs for the above measures, suggest that the use of capnography may be more effective as a rule-out rather than a rule-in test for detecting adverse events. However, this interpretation needs to be balanced with findings forthcoming from the aggregate LR values. It is recommended that in order ‘to alter clinical management’ a positive LR >10 and a negative LR <0.1 are desirable.31 The corresponding aggregate values for this review (1.89 and 0.30, respectively) and indeed, the corresponding upper and lower 95% CI limits of 2.34 and a 0.12, respectively, fall short of the standards of this recommendation. A further measure of diagnostic test quality is the separation ratio positive LR/negative LR between the two likelihood ratios. In the literature,32 the criterion that this ratio lies below 50 has been used to characterise a weak diagnostic test. For the current review, the corresponding value is 6.3. Clearly, therefore, the values of the LRs for this study are unsupportive of capnography as a useful diagnostic test.
There was a lack of evidence for a statistically significant difference in the number of adverse events detected when capnography was used in addition to standard monitoring (48.8% (95% CI 32.9 to 64.9)) compared with chance alone (50%). Lastly, across the three studies used to assess physician interventions (table 3), there was considerable variation in the proportion of physician interventions in response to abnormal capnography readings. Also, the 95% CIs for these proportions were very wide, undermining the accuracy of the above sample proportions as estimates of the true or population proportion of physician interventions based on capnography monitoring.
The clinical significance of these findings may be limited for a number of key reasons:
▸ It was not feasible to perform a meta-analysis relating to physician intervention due to small patient numbers. Indeed, only three of the studies included reported data relating to physician intervention. The inability to simultaneously address adverse event detection and the link with physician intervention for each study identifies an area requiring further research.
▸ There was high variability in the definition of an adverse event, ranging from severe oxygen desaturation (<75% at any time) to transient hypoxia (<93% for 15 s).
▸ The restricted use of supplemental oxygen in six of the studies. Despite high flow oxygen being recommended in all sedated patients by the joint Royal College of Emergency Medicine/Royal College of Anaesthetics statement,2 this was not routinely delivered in any of the studies included herein. The potential benefit of capnography may well be reduced in well-oxygenated patients.
▸ Depth of sedation is an important determinant of safety during PSA and none of the studies specified an intended depth of sedation in either their methods or outcomes.
▸ There was an element of incorporation bias as some of the studies used capnography or standard monitoring as part of the definition of an adverse event, which could lead to an overestimation of diagnostic accuracy.
▸ Importantly, there is consistency in the ability of capnography to detect adverse events, including hypoxia, before other monitoring across different clinical settings.33 There are several possible factors which may account for the absence of a clear relationship between capnographic detection of adverse events, physician intervention and improved outcomes. These include the choice of research questions, the choice of study design, a lack of study power and the rarity of significant adverse events. Thus, it is as yet unclear that the above study findings provide evidence of absence of improved patient safety.
Quality of the evidence
This is the first systematic review to evaluate capnography use during PSA in adults specifically within the ED. The strength of this review is based on its methodology. Several methods were used to reduce publication bias, including comprehensive literature searching, implementing strict prespecified inclusion and exclusion criteria, screening all papers by two reviewers to reduce selection bias and performing a thorough quality assessment.
The main limitation is that there was only one published RCT studying capnography as a primary intervention.25 While the existence of a common treatment outcome supports use of meta-analysis, as with many reviews, study design and outcome definitions were not homogeneous across studies, resulting in statistical heterogeneity. For the meta-analyses in this review, there was estimated to be moderate between-study statistical heterogeneity or in the case of the sensitivity and negative likelihood ratio for capnography detecting an adverse event, high between-study heterogeneity. Statistical heterogeneity can arise from clinical heterogeneity, random errors and errors in estimation of within-study variability.34 As this study included cohort studies, possible sources of clinical heterogeneity included differences in clinical methodology between studies, such as the drug regimen, frequency of ETCO2 monitoring and the definition of an adverse event. Using the random-effects model, we sought to explore clinical heterogeneity from the above sources. Although attempts to reduce publication bias were performed via comprehensive literature searching, the small number of studies per outcome precluded formal assessment of publication bias via a funnel plot,29 which would have provided increased rigour.
Current guidelines advocate the routine use of capnography during PSA.2 ,4 The potential benefit of capnography during PSA has been eloquently debated elsewhere.3 ,35 Determining whether or not its use provides additional safety is difficult because it is unclear what defines an optimal safety measure and how this measure might alter clinician practice to prevent adverse events. In the RCT studying capnography as a primary intervention, the addition of capnography reduced the number of patients with hypoxia. However, the more important decision to intervene during procedural sedation is multifactorial, involving both human and clinical factors. Part of this issue is the immense variability in sedation-related adverse event reporting. The ‘Adverse Events Sedation Reporting Tool’ has recently been developed by the World Society of Intravenous Anaesthesia36 and has already been successfully trialled.37 ,38 This tool includes a description of the adverse event and its severity, the interventions performed and the outcome.
Across all studies in this review, capnography measurements and oxygen saturations were used to identify adverse events and also abnormal measurements were included as one of the variables used to define an adverse event. Thus, in order to consistently report and evaluate the efficacy of capnography in reducing adverse events and improving patient safety, there is a clear need for a standardised tool such as that advocated by the Word Society of Intravenous Anaesthesia.36
Conclusion
This review demonstrates that while, in adult patients undergoing PSA in the ED, capnography may be able to distinguish between patients who will and will not subsequently experience an adverse event, there is a lack of statistical evidence to support its clinical usefulness as part of routine care during the procedural sedation of adults in the ED. Further, there is insufficient evidence to suggest that the addition of capnography to standard monitoring increases patient safety. Despite this, given the diagnostic utility of capnography identified in this and a previous review,3 its ease of use, low cost and lack of risk, we advocate compliance with current professional guidance on the use of capnography in the ED during PSA. However, there is a need for well-powered RCTs employing an accepted adverse event reporting tool while simultaneously quantifying physician likelihood to intervene during PSA in the ED. Such studies, along with thorough cost–benefit analyses, are required to substantiate professional guidelines and determine whether there is real clinical benefit from using capnography during PSA in adults in the ED.
References
Footnotes
Contributors AG conceived the study. CD drafted the protocol, carried out the study selection, extracted data and analysed the studies. RB and AG arbitrated the study selection. MD reviewed the statistics of the meta-analysis. GL reviewed the quality and accuracy of the study. All authors contributed to the revision of the manuscript. CD takes responsibility for the paper as a whole.
Competing interests None declared.
Ethics approval NHS ethical review was deemed unnecessary. This study met ethical approval by The College of Medicine and Veterinary Medicine, University of Edinburgh, UK.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement All data collected in the review has been published within the article or supplementary files online.