The utility of any clinical sign is a function of both its ability to detect or predict a particular condition, and its reproducibility between different observers. In emergency paediatrics an informed consensus has been reached on the clinical parameters defining shock.¹ These include the presence of tachycardia, hypothermia, or hyperthermia, decreased mental status, signs of impaired perfusion (flash capillary refill and bounding pulse (warm shock) or delayed capillary refill time >2 s, cold peripheries and decreased pulse volume (cold shock), and urine output <1 ml/kg/h in a child with suspected infection.¹ Readily identifiable features of shock (capillary refill time, temperature gradient, pulse volume, and signs of dehydration) have been incorporated into advanced life support guidelines as part of the rapid, structured cardiopulmonary assessment of critically ill or injured children.^2,3 As such they are used to prioritise initial management. However, information supporting the reliability of these clinical signs to define shock is lacking, in particular data examining the robustness of these signs between different observers. The interpretation of these features therefore requires more detailed assessment, since their presence is often the sole basis on which emergency interventions are initiated. The current study sought to examine prospectively the inter-observer reproducibility of bedside clinical features of shock. It did not, however, seek to validate the ability of any sign to define shock.

PATIENTS AND METHODS

RESULTS

One hundred consecutive paediatric admissions were assessed independently by each of the four clinicians. The study group age ranged from 2 days to 10 years 11 months. Presenting complaints included fever (n = 78), cough (n = 43), respiratory distress (n = 25), diarrhoea and/or vomiting (n = 26), and convulsions (n = 25). Many had poor nutritional status: undernutrition (WAZ score −2 to −3 SD) and severe malnutrition (WAZ score <−3SD, plus visible severe wasting) were present in 22% and 18% respectively, and seven children had oedematous malnutrition (kwashiorkor). Final diagnoses are shown in table 2⇓, which were typical of the usual diagnostic spectrum in children admitted to KDH. Three children subsequently died (a 1 month old infant with gastroenteritis and LRTI, and two children with malnutrition, aged 16 and 32 months, respectively), one child absconded (LRTI), and two were transferred for surgical intervention.

Table 3⇓ shows the level of agreement for all features assessed by the study team. Overall agreement for CRT was moderate (κ = 0.42), and was better for normal values (⩽1 second) (κ = 0.48) and clearly abnormal values (⩾4 seconds) (κ = 0.49). There was moderate to substantial agreement between observers for temperature gradient, being slightly better for the lower limb (κ = 0.62) than the upper limb (κ = 0.57). There was moderate agreement in the assessment of weak pulse volume (κ = 0.40); however, there was little to no agreement for bounding pulse volume (κ = −0.01). In the assessment of hydration status the level of agreement was substantially better for a decreased skin turgor (κ = 0.55) than either sunken eyes or dry mucous membranes, for which agreement was only fair (0.34 and 0.39 respectively). There was no significant difference in these findings after stratification for the presence or absence of malnutrition.

DISCUSSION

We have examined the between-observer agreement in medically qualified workers for the assessment of clinical signs of cardiovascular compromise, that are currently recommended by current guidelines for the assessment of critically ill and injured children.^2,3 We found that agreement was substantial for the assessment of temperature gradient (in the lower limbs): high moderate agreement for upper limb temperature gradient and low moderate agreement for the assessment of weak pulse volume. Agreement for capillary refill time was greatest for normal values (1 second) and significantly delayed refill time (⩾4 seconds), but only fair for other values. In the assessment of hydration status the best agreement between observers was for reduced skin turgor. Sunken eyes and decreased skin turgor are also non-acute features of severe malnutrition, and in the presence of severe malnutrition the assessment of hydration status is difficult.^7,8 In this study however, the interrater agreement for signs of dehydration was similar in children both with and without anthropometric features of severe malnutrition. The more important question would have been, with regard to sign recognition in the malnourished group, an appraisal of the validity of the assessments, but this was beyond the scope of this study.

Basic and advanced paediatric life support (APLS) and American College of Critical Care guidelines recommend the use of readily obtainable features of cardiovascular compromise (delayed capillary refill time, poor pulse volume, and temperature gradient) to determine initial management, in particular the use of resuscitation fluids.^1–,³ Furthermore, both clinical and biochemical features of compensated shock (hypovolaemia) are also recommended to define the severe inflammatory response syndrome (SIRS).⁹ In settings where resources allow, invasive monitoring and quantification of the biochemical derangements (blood gas analysis, urea, creatinine, and lactate estimation) provide supportive evidence for hypovolaemia. However, in resource-poor areas of the world clinical assessment is often the sole determinant of a child’s initial management. It is important to provide information on just how reliable bedside features of shock really are. Capillary refill times and assessment of pulse volume recommended by the recent American consensus guidelines¹ achieved only a low moderate to poor interrater agreement, questioning the reliability of such parameters.

Nolan and colleagues have shown that initial triage and emergency treatment are poor in most district hospitals and in many teaching hospitals in developing countries. Inadequate staff training and the absence of guidelines for standard management were identified as contributing factors.¹⁰ We found, despite a period of training in sign recognition, that inter-observer agreement over a wide range of clinical conditions was moderate for most features assessed and only substantial and high moderate for the assessment of temperature gradient, indicating that, for clinical signs with poor interrater agreement, either training was inadequate or more plausibly that these features are too subjective to be amenable to accurate definition.

Emergency triage assessment and treatment (ETAT) guidelines were designed with the aid of the WHO for use in developing countries.¹¹ The emergency signs used to define shock are delayed capillary refill time (≥4 s), cold hands, and weak, fast pulse, and indicate the urgent need for volume expansion. Evaluation in 3837 Brazilian children attending the emergency department, identified shock in only four children (0.1%), all of whom died.¹² In a separate study in 2281 Malawian children attending casualty, delayed capillary refill time (>3 s) and cold hands were ascertained by both a doctor and nurse in only five and 13 patients respectively.¹³ These data suggest that these criteria are insensitive, since in both studies shock was rarely diagnosed (<0.5%), although the case fatality in those defined as shocked approached 100%. In our hospital on the coast of Kenya, clinical assessment for shock has been part of our routine admission practice for a number of years. A review of data from children admitted in the year 2002 found delayed CRT (>3 s) was diagnosed in 8% (case fatality = 24%), a temperature gradient described in 4% (case fatality = 12%), and weak pulse volume in 3% (case fatality = 23%). Case fatality in children without any of these features was 2.6%; suggesting that even when routinely used by medical personnel (with varying degrees of clinical experience), these signs have some value in being able to identify a group with poor prognosis in whom volume resuscitation should be considered.¹⁴ Furthermore, these data also suggest that shock is much more common among routine admissions to the general paediatric ward than previous studies in the developing world have indicated.

Published literature regarding the inter-observer variation in triage signs is limited,^13,15 and is especially lacking for clinical signs of shock. In the current climate of clinical audit, it is surprising that inter-observer variation studies have not been more widely used to validate clinical practice. A general appreciation of the need for training of basic sign recognition is fundamental to improving triage and initial management. In developing countries a culture of early recognition of compensated shock and the institution of simple, potentially life saving treatments may improve outcome. This study provides reasonable evidence of the reproducibility of some of the bedside clinical features currently used in the assessment of shock, in particular, for lower limb temperature gradient and delayed capillary refill ⩾4 seconds in the assessment of shock and for skin turgor in the assessment of hydration status. Poor inter-observer agreement in the assessment of pulse volume and other features of dehydration questions the reliability of these signs. Further studies should be conducted to examine the validity of these signs and other less subjective features of shock (such as tachycardia and tachypnoea), but these are unlikely to be within the scope of a busy tropical paediatric unit.