Objectives The sensitivity and specificity of consensus triage criteria for identifying which apparently inebriated patients could be triaged to care in a sobering centre were determined. Sensitivity and specificity for modifications to these criteria were also investigated.
Methods Paramedics prospectively collected data on apparently inebriated persons en route to the emergency department (ED). 99 of these patients' ED charts were retrospectively reviewed to assess who actually required ED care.
Results Of 99 subjects with both paramedic and ED chart data available, most were male (89%), homeless (57%) and found on the street (81%). Five were admitted and 13 others appeared to require ED care. Per consensus criteria, only 40 were eligible for triage to a sobering centre, but among those were five who appeared to require ED care (sensitivity 72%, 95% CI 47% to 90%; specificity 43%, 95% CI 32% to 55%). Paramedic opinion alone was specific (80%) but not very sensitive (39%). Lowering the pulse exclusion threshold from 130 to 83 would increase sensitivity to 100%, but decrease specificity to 22%. A simple post hoc rule excluding those with age >55 or pulse >83 from non-ED care had high sensitivity (94%) and fair specificity (61%). The consensus criteria's sensitivity and specificity varied (65–83% and 44–49%, respectively) depending on which ED services were considered optional (eg, psychiatric consultation, ECG, intravenous fluids, etc.).
Conclusion Most apparently inebriated individuals in this study did not require ED care, but prospective identification of these persons is difficult. A low exclusion cut-off for tachycardia may improve sensitivity.
- emergency department
- clinical assessment
- emergency care systems
- mental health
- alcohol abuse
- cost effectiveness
Statistics from Altmetric.com
- emergency department
- clinical assessment
- emergency care systems
- mental health
- alcohol abuse
- cost effectiveness
The Institute of Medicine Committee on the Future of Emergency Care in the US Health System has described a national crisis of emergency department (ED) overcrowding.1 From 1996 to 2006, the annual number of ED visits increased from 90.3 million to 119.2 million (up 32%), while the number of EDs decreased from 4019 to 3833 (down 4.6%). The number of visits to the ED in which the patient was transported by ambulance also increased 17% from 1997 to 2006.2 ED overcrowding leads to decreased quality of care, increased mortality and decreased access to emergency and speciality services.3 4
Identification of non-emergency cases that could be dealt with outside the ED may improve efficiency. Some 620 000 annual ED visits are made for a primary diagnosis of acute alcohol intoxication. A half to two-thirds appear to be uncomplicated, and might safely be triaged to management in non-ED settings.5 In several US cities, facilities for caring for inebriated patients with non-acute medical needs have been created—so-called ‘sobering centres’. In the city where this study took place, a collaborative effort by public and private organisations created such a centre in 2003. Patients are transported to this facility by ambulance, via the police or by a mobile assistance patrol van staffed by non-medical personnel. This sobering centre has 20 beds and is staffed by a multidisciplinary team of nurses, social workers and caseworkers.
Some patients who initially may appear merely inebriated, however, are found to have a life-threatening condition masquerading as or co-occurring with alcohol intoxication.6–9 How often this occurs, and whether or not these patients can be identified prospectively by paramedics, remains unclear.
Our aims in this study were to estimate the sensitivity and specificity of triage criteria developed by consensus to identify which apparently inebriated patients might safely be triaged to non-ED care—that is, a sobering centre. We also investigated the sensitivity and specificity of modifications to these criteria.
We conducted a retrospective cohort study by review of medical records. Our institutional review board approved the study. Because data were collected only from medical records, we obtained a waiver of informed patient consent.
Population and setting
This study took place in an urban area with a population >500 000. We studied records of patients who had been transported to an ED during July 2003. Paramedics completed a paper survey for each patient in which alcohol intoxication appeared to be a primary factor leading to transport (n=172). We located ED charts for 99 of these patients at nine different EDs.
Methods of measurement
For each of these patients, paramedics recorded demographic characteristics, location from which the patient was transported, chief complaint, vital signs, Glasgow Coma Scale score, finger-stick blood glucose level, presence or absence of a number of subjective and objective exclusion criteria, whether or not the patient officially met criteria for triage to the local sobering centre and whether or not the paramedic thought such triage was appropriate. Consensus triage criteria were embedded within the survey form as ‘exclusion criteria’ checkboxes. These patients were all transported to a San Francisco ED per long-standing local emergency medical services (EMS) protocol.
The retrospective review was conducted using a standardised data collection form to assess which of the subjects transported actually required ED care. Information obtained from this collection process included demographics, tests performed, diagnoses, and disposition of the patient. Data on 99 patients were entered into a computerised database for analysis by San Francisco Department of Public Health quality assurance personnel. Patients who were admitted to the hospital after transport, who received speciality consultation, had positive radiology results, required suturing of a laceration or were given charcoal lavage were considered to have required ED care. Persons who had negative radiological examinations or who received only non-emergency treatment were not considered to have required ED care.
Primary data analysis
We estimated sensitivity and specificity for four sets of triage criteria that would have excluded patients from non-ED (ie, sobering centre) care:
Consensus exclusion criteria (see table 1). These criteria were generated by consensus of local physicians and EMS personnel.
Paramedic opinion—that is, response to the question, ‘Do you think transport to the sobering centre would be appropriate for this patient?’
Purely data-driven exclusions (subject age >55 and pulse >83 only).
Consensus exclusion criteria with a data-driven modification (excluding those with pulse >83 from the sobering centre).
Sensitivity was further analysed by varying assumptions about which ED services constituted required services. Sensitivity, specificity, and negative and positive predictive values were calculated with 95% CIs.
Some ED data was missing for 73 subjects, and they were excluded from subsequent analysis. However, demographic data were available for all 172 subjects, and analysis indicates that the missing cohort did not vary significantly in any demographic measures when compared with the 99 subjects for whom we had all data. The 99 subjects studied were mostly male (89%), homeless (57%) and found on the street (81%). A substantial percentage of these subjects (42%) were identified by paramedics as frequent users of EMS (table 2).
Eighteen subjects (18%) appeared to require ED care—these included five hospitalised subjects, and 13 subjects who received specific ED services but who were not admitted to the hospital (see figure 1; one subject required two specific ED services). Forty-two patients were discharged from the ED after negative ECG or radiological studies (n=20), or after non-emergency treatment (n=22). The other 39 patients were observed and then discharged from the ED (table 3).
Table 4 summarises the sensitivity and specificity of the four criteria. Consensus criteria were relatively sensitive (72%) but not very specific (43%), while paramedic opinion alone was specific (80%) but not very sensitive (39%). Use of paramedic opinion alone would have mistriaged 3 of 5 patients who were subsequently admitted, and 8 of 13 who required ED services. To investigate ways of improving consensus criteria, we generated additional, post hoc exclusion thresholds from this sample. Lowering the pulse exclusion threshold from 130 to 83 would increase sensitivity to 100%, but decrease specificity to 22%. A simple rule which merely excluded those with age >55 or pulse >83 from non-ED care had high sensitivity (94%) and fair specificity (61%).
For consensus criteria, sensitivity and specificity varied (65–83% and 44–49%, respectively) depending on which ED services (eg, psychiatric consultation, ECG, intravenous fluids, etc.) were considered optional. Sensitivity was highest (83%, 95% CI 52% to 98%; specificity 44%, 95% CI 32% to 55%) if only admission and laceration suturing were required services, and was also relatively high (80%, 95% CI 52% to 96%; specificity 44%, 95% CI% 33 to 55%) when psychiatric consultation was not considered to be required. Specificity was highest (49%, 95% CI 32% to 65%, sensitivity 65%, 95% CI% 52% to 77%) if all services including complete blood count, comprehensive metabolic panel or other blood tests were deemed to have been required (see table 5).
Five patients would have been mistriaged by consensus criteria, and might have bounced-back to the ED had they actually been transported to the sobering centre. Three of these five patients required mental health assessment and treatment (table 6). Two of these patients had been identified by paramedic opinion as appropriate for transport to the sobering centre.
Negative predictive value for consensus criteria was 88%, but this figure may not be generalisable because the true prevalence of need for ED care among persons who are apparently inebriated may differ significantly, depending on setting.
In our sample, most apparently inebriated individuals did not require defined ED services or admission to the hospital, but prospectively, safely and efficiently identifying these persons is challenging. Such identification is markedly difficult because it is done in the field, where decisions are made with limited information. However, the initial encounter with these patients occurs at a time when relatively few resources have as yet been devoted, and when appropriate triage has greatest efficacy.
Our consensus criteria's sensitivity of 72% means that out of 100 patients who actually required ED care, we could expect 28 to be undertriaged (ie, an underestimation of the severity of an illness or injury was made) and sent to a sobering centre. What undertriage rate is acceptable in this population? For general medical conditions dealt with by EMS, rates of reported undertriage have varied. A recent prospective study of a prehospital EMS protocol for triaging patients to alternative transport reported that 10% of subjects triaged to transport by taxi were ultimately admitted to the hospital.10 The authors concluded that use of such a protocol remains ‘an unproven concept’. Schmidt et al11 reported a 9% undertriage rate, and noted that patients with psychiatric complaints were at highest risk for undertriage. In a comparison of triage decisions made by paramedics versus emergency physicians, Pointer et al12 reported a 9.6% undertriage rate, and called this rate ‘unacceptably high’. Asplin (commenting on the Pointer study) noted the difficulty of establishing an acceptable undertriage rate: ‘Although it is easy to agree that undertriaging 10% of patients is unacceptable, when would the sensitivity be high enough for us to consider paramedic triage safe? I do not have the answer to that question, but it seems clear that, if we were finally able to reach an ‘acceptable’ sensitivity (ie, an acceptable degree of undertriage), the specificity would be so low that the potential for saving system resources would be marginal at best.’13 The American College of Surgeons Committee on Trauma indicates that 5% undertriage is acceptable (and is associated with 25–50% overtriage).14 Participants in a symposium convened to assess EMS triage (the Neely Conference) were unable to reach consensus on an acceptable undertriage rate.15
Even after patients are seen and treated in the ED, some bounce-back occurs. The 2006 US National Ambulatory Care Survey2 reported a 3.6% return rate for patients seen in the same ED within 72 h, and a study in Taiwan16 reported a 72 h return rate of 5.5%. Keith et al17 reported a rate of 2.2% unscheduled return to the ED within 72 h, whereas another study found a 0.4% unscheduled return visit rate.18 ED bounce-back may occur more often when patients present with mental illness or problems of drug misuse. In a study looking at risk factors for admission to the hospital within 72 h after an ED visit, patients with mental health disorders had a higher risk of return than with any other diagnoses, and 71% of those cases were alcohol related.19 Mental health issues were a factor in three of five cases in our study in which use of consensus criteria would have mistriaged to non-ED care.
Higher undertriage rates might be more acceptable given an alternative other than simply non-transport to the ED—for example, transport to alternative medical care. At the San Francisco sobering centre, patients are cared for by nursing staff following a set of established protocols for monitoring, treatment, referral or return to the ED. For example, out-of-range vital signs mandate transport to the ED. During a study period in August 2003 to January 2004, 5% of patients at this sobering centre were discharged to an ED. Higher undertriage rates place an additional burden on an alternative care centre to be able to accurately recognise, possibly perform additional care and arrange for transport back to an ED.
Modifying our consensus criteria by broadening the range of exclusionary vital signs could provide greater safety. Lowering our pulse rate exclusion cut-off improved sensitivity but, as noted, this lower exclusion cut-off (pulse >83) was generated post hoc from our data and may not be generalisable to other populations. Five subjects who may have bounced-back to the ED in our study would have been appropriately initially triaged with use of a lower exclusion cut-off for tachycardia (eg, pulse >100, rather than the consensus exclusion criteria of pulse >130).
Our consensus criteria's specificity (43%) means we could also expect significant overtriage—that 57 patients out of 100 who did not need ED care would still be sent to the ED. In addition to resource waste from overtriage, there are other risks—for example, exposing patients to unneeded medical tests, procedures and infections in medical facilities. In our study, paramedic opinion would have yielded the least overtriage of any of the four criteria studied, providing relatively high specificity (80%). Of 39 subjects in our study who were discharged after only observation in the ED, paramedic opinion alone would have prospectively identified all but three. Many subjects (42%) in our sample were identified as ‘frequent callers’, and over time paramedics may have become familiar with their medical histories and circumstances. Judicious use of paramedic experience to further inform triage decisions based on objective criteria might improve specificity while still providing reasonable sensitivity.
However, use of paramedic opinion alone would have resulted in substantial undertriage (sensitivity 39%). This finding is consistent with results from other studies showing that paramedic opinion alone can be inaccurate in triaging patients.20–23 In a meta-analysis of studies reporting US paramedics' ability to determine medical necessity of ambulance transport, Brown et al24 concluded that insufficient evidence exists to support paramedic determinations of medical necessity for ambulance transport, reporting a likely undertriage rate of 9% but noting that the rate could have been as high as 29%. Accurate triage based on opinion alone appears problematic across skill levels. For example, Patterson et al investigated inter-rater agreement in a small sample of physicians giving opinions on the necessity of EMS transport based solely on ED diagnoses. Overall agreement among these physicians was classified only as ‘fair’ (κ=0.31), and was notably low when considering mental illness diagnoses (κ=0.14).25 Another study reported only ‘fair’ agreement (κ=0.38) among ED nurses, emergency physicians and family physicians on the need for ‘urgent care’ for ED patients.26 In a study in which an emergency physician and an internal medicine physician reviewed the same 219 cases and were asked, ‘Considering all aspects of this case, was this an emergency?’, the internist indicated 36% of the cases were emergencies, while the emergency physician identified 90% of the cases as emergencies.27 The two physicians agreed more often (91% and 77%, respectively) when asked which cases could be treated in non-ED settings.
Complicated triage criteria may place a burden on EMS staff, and our consensus exclusion criteria required accurately determining the presence or absence of a number of conditions. In one case, the paramedic simply failed to check a box on the exclusion criteria form for ‘suturable laceration’ in a patient with this condition. Interestingly, in our particular sample we found that simplification via merely using cut-offs for age and pulse provided sensitive and fairly specific triage. Nevertheless, these simple rules would still require supplemental criteria (eg, subjects with serious cardiac or gastrointestinal disease, cervical spine injuries, etc., might be mistriaged using such simple rules). At the risk of complicating triage, a combination of specific criteria, vital signs limits and judicious use of systematised paramedic knowledge of particular patients may provide reasonably safe triage.
Use of any of these criteria is ultimately designed, in part, to increase efficiency and reduce overtriage. The use of consensus criteria over 1 month would have diverted 71 patients to the sobering centre instead of to more expensive care in the ED. The cost of ED care for acute alcohol intoxication is significant. In 2006 the average cost of an ED visit in the USA was US$638.28 Thus, the diversion of 71 subjects to non-ED care represents substantial direct savings. Not reflected in these figures, however, are additional indirect savings in the form of reduced ambulance diversion, better EMS response times and reduced impact on the ED. Nor is the ED the optimal setting for acute intervention in alcohol abuse or dependence, where encounters are characterised by a lack of time and privacy, ignorance about alcohol intervention by the ED staff (eg, unfamiliarity with screening instruments, unawareness that brief interventions can be beneficial) and limited reimbursement for screening or intervention.29 As few as 15% of ED patients with obvious alcohol problems ever have their drinking behaviour addressed while in the ED or through an ED referral.30 31
Future research might focus on ways these criteria might be improved by more formal processes of triage development—for example, by the Delphi method, a structured process to elicit expert knowledge using a repeated series of questionnaires to gather and provide information that continues until consensus emerges.32 An examination of whether such criteria might be generalisable for triage of persons who appear to be misusing other drugs of abuse, for example opiates or stimulants, might be useful. EMS would benefit from investigating ways to elucidate and track triage outcomes, especially as a way of improving paramedic education. Ironically, paramedics may receive negative feedback more often about cases they overtriage than those they undertriage simply because their transported patients are available for subsequent evaluation. Ideally, EMS agencies will identify some repeat calls as undertriage and provide paramedic feedback, but information is undoubtedly lost across agencies or over time. A potential advantage of triage to non-ED care is that outcome can be tracked for more patients and thus better inform paramedics about their triage decisions. Are there other tests that could be administered safely and efficiently by EMS field personnel to assist in triage—for example, breath ethanol levels or urine toxicology? Should field triage criteria be restricted to use in patients within certain age ranges, and without certain known past diagnoses or symptom categories? Should protocols be tailored to different EMS skill levels (eg, basic life support vs advanced life support responders)? Would early consultation and communication with physicians by EMS personnel in the field improve triage? From both clinical and public policy perspectives, further analysis of what constitutes reasonable undertriage and overtriage rates would be beneficial.
Our sample was taken from an urban area, and subjects were mostly male and homeless. Thus, our results may not be generalisable to suburban or rural populations. Subjects were studied in a single summer month, and it is possible that a study during other seasons might alter the results. Weather in the geographical area studied is relatively moderate all year round, although there is usually substantial rain in winter months. It is unknown how many subjects, if any, refused transport or care and whether these patients represent an important subgroup. We could not locate ED charts for 73 of the initial 172 persons transported by paramedics; this loss to follow-up could bias our results if it was not random (the demographics of the 73 subjects with missing ED charts did not vary significantly from those of the 99 subjects studied). The use of hospitalisation and the specific ED services we chose may not be the best standards for determining the necessity of ED care—for example, some patients who legitimately need transport to the ED do not require hospitalisation, but might need diagnostic investigation. Some of the consensus criteria are vague and open to interpretation (eg, how recent is ‘recent head trauma?’; what constitutes a ‘suturable laceration?’). Paramedics were not blinded to their participation in the study, and this possibly influenced their opinions on the appropriateness of triage to the sobering centre (eg, they may have been more conservative in their opinions, leading to an underestimation of the undertriage rate for paramedic opinion). Specific modifications to consensus criteria (eg, pulse <83) were completely data driven from this limited sample, and may not be generalisable. Finally, our sample size was not large enough to reliably capture rare dangerous conditions that might masquerade as alcohol intoxication (eg, subdural haematoma), and it is unclear how sensitive triage criteria would be for detecting such conditions. Nevertheless, our main finding (that prospective triage is difficult) is unaffected by this limitation.
We found that prospectively identifying inebriated patients who can be diverted to non-ED care is difficult, but potentially provides improved efficiency. Actual field use of our consensus criteria would have resulted in substantial undertriage. The acceptability of this rate of undertriage (given, for example, that patients would have been diverted to alternative medical monitoring and care) or of other undertriage rates remains unclear. A lower exclusion cut-off for tachycardia in our studied criteria may improve sensitivity.
We are grateful to our anonymous reviewer for excellent comments and suggestions.
Funding Supported by NIH DA018179.
Ethics approval This study was conducted with the approval of the University of California, San Francisco, Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.