Send letter to journal:
Re: AMPDS - As good as it gets?

Dear Editors,

Professor Deacon and colleagues report the precision with which the Priority Dispatch Corporation’s ProQA™ call interrogation software detects Acute Coronary Syndrome (ACS) amongst ‘999’ calls from the Southampton area (1). They analysed an 8 month sample of 42,657 emergency calls and identified 3368 patients with a ‘chief complaint’ of ‘chest pain’ as determined by the use of ProQA™ . The authors were then able to access the MINAP (2) data set for Southampton General Hospital and identify which patients subsequently had an ACS confirmed. Cross referencing these two data sets revealed 263 patients with a ‘proven’ ACS, 187 of which had been ‘correctly’ identified.

On the basis of these results, the authors calculated that the sensitivity of ProQA™ in detecting ACS in this population was only 71% with a specificity of 92.5% and a positive predictive value of 5.6% (95% CI 4.8 to 6.4%). The results, as I understand them, are reproduced in table 1 (there are typographical errors in the published table). The authors conclude that the sensitivity and positive predictive value of ProQA™ “does not enable accurate identification of patients with ACS”. There are three points that merit further discussion which may fundamentally influence our interpretation of these results.

Firstly, the 95% confidence interval for the sensitivity of ProQA™ (the proportion of people with ACS who are identified) lies between 66 and 77%. This sensitivity is based on the true positives – the 187 patients with both chest pain and ACS. However, it may not be wholly appropriate to use the ‘chest pain’ chief complaint determinant alone as a measure of the performance of ProQA™. The software allows additional subdivision of all calls according to clinical urgency. The authors allude to this when they indicate that 230 of the 263 patients with confirmed ACS had actually been categorised as having an immediate threat to life by ProQA™ – thus triggering an urgent (Category A) response – regardless of whether chest pain was the chief complaint. The sensitivity of ProQA™ in identifying patients with ACS who could benefit from a rapid response could therefore be as high as 87% (230/263) with 95% CI between 83% and 92%.

Secondly, for rare events, the major determinant of predictive value is the prevalence of the condition in the population tested. No matter how specific the test is, if the population is at low risk of having the disease, positive results are more likely to be false positives and the predictive values will be low. In this study, the apparent prevalence of ACS is 0.61% (263 / 42657) which, although three times higher than a general population estimate (3), appears low for a sample of 999 calls. An argument could be made that it is the denominator of 42,657 which causes the problem here. This number could be reduced by excluding the large number of calls that would clearly have been of no relevance to ACS (e.g. minor wounds and injuries). Reduction of the denominator value would increase prevalence and positive predictive value. The likelihood ratio is less likely to be influenced by prevalence and it probably better reflects the complexity of the ProQA™ process. For this data set, a positive ProQA™ result would be eight to ten times as likely to be seen in someone with ACS as opposed to someone without ACS (Likelihood = (sensitivity/(1- specificity)) = 9.5 with 95% CI: 8.7 to 10.3).

Thirdly, the trade off between acceptable sensitivity and specificity requires us to weigh the consequences of missing ACS (a false negative) against the consequences of erroneously dispatching resources (a false positive). In making these judgements, the context of this study is important. Accurate and rapid deployment of thrombolysis capable paramedics is considered key to the pre-hospital management of ACS and the efficiency of such deployment is retrospectively scrutinized by local, regional and national authorities. The emphasis is therefore on high sensitivity of any initial call handling system. The higher the sensitivity, the greater the ACS detection rate and the lower the false negative rate.

Sensitivity tends to be favoured at the expense of specificity when the penalty associated with missing a case is high – as is often argued for ACS. On the other hand, specificity should be favoured relative to sensitivity when the cost or risks associated with a false positive are high. One might argue that in a climate where it is still the clinician who determines actual treatment (not ProQA™), Ambulance Services are not the last line of defense, most hospital Coronary Care Units and Emergency Departments have clearly defined rapid assessment and treatment pathways for ACS and thrombolysis capable paramedics are still a relatively scarce resource, we should favour a caller interrogation system with a high specificity and low number of false positives. If this is the case, a sensitivity of up to 77% (and possibly as high as 92%) and a specificity of up to 93% (upper 95% CI) may be perfectly reasonable – and as good as it gets!

Roderick Mackenzie

References

1. Deakin CD, Sherwood DM, Smith A, Cassidy M. Does telephone triage of emergency (999) calls using advanced medical priority dispatch (AMPDS) with Department of Health (DH) call prioritisation effectively identify patients with an acute coronary syndrome? An audit of 42 657 emergency calls to Hampshire Ambulance Service NHS Trust. Emergency Medicine Journal 2006;23:232-235.

2. http://www.rcplondon.ac.uk/college/ceeu/ceeu_ami_home.htm (accessed on 17 April 2005).

3.National Institute for Clinical Excellence. Technology Appraisal No. 47. Guidance on the use of glycoprotein IIb/IIIa inhibitors in the treatment of acute coronary syndromes. September 2002.