Jump to comment:

• Re: Authors' response
  Peter McQuillan
  Published on: 29 April 2016
• Appropriate analysis and reporting of cluster randomised trials
  Steve Goodacre
  Published on: 29 April 2016
• In need of a PEA?
  Andrew M. Mason
  Published on: 29 April 2016

• Published on: 29 April 2016

  Re: Authors' response

  • Peter McQuillan
  • Other Contributors:
    • Bernard Higgins

  Dear Editor

  We must accept that our original analysis, which assumed statistical independence between observations obtained from staff within the same hospital, might not be justified. To explore this possibility we have computed Intra Cluster Correlation Coefficients (ICCs) using estimated components of variance obtained from an analysis of variance in which hospitals were treated as random effects within a nest...

  Show More

  Dear Editor

  We must accept that our original analysis, which assumed statistical independence between observations obtained from staff within the same hospital, might not be justified. To explore this possibility we have computed Intra Cluster Correlation Coefficients (ICCs) using estimated components of variance obtained from an analysis of variance in which hospitals were treated as random effects within a nested sampling design.

  With regards the total score at sixty seconds the between hospital component of variance was negative and hence the estimated ICC was set to zero. The ICCs and variance inflation factors (VIFs, assuming an average cluster size of 15) for all four outcome measures are presented below:

   

  Outcome	Original p-values	ICC	VIF	Adjusted p-values
  Score @ 60 seconds	0.034	0.000	1.00	0.048
  Total score overall	0.067	0.011	1.15	0.095
  Proportion with max score at 60 seconds	0.054	0.002	1.03	0.077
  Proportion with max score overall	0.100	0.016	1.22	0.123

  As pointed out, the consequences of positive ICCs is that the reported p- values, which ignored the clustering effect, will tend to be biased downwards. A subsequent analysis, which adjusts for clustering within the study, produced elevated p-values for all outcomes with that for the score at 60 seconds remaining significant at the 5% level.

  We did however, state in the paper that the results were at best of marginal significance, statistically. The ceiling of a maximum of 8 correct causes may have reduced the ability to demonstrate a significant effect, if one exists. Despite these p value discussions, the paper remains of importance for two reasons. Firstly, it points out that despite the best of intentions, the use of a device to augment recall may potentially lead to adverse effects; 78% house officers could recall hypothermia, which in UK an uncommon cause with a long treatment wheelbase, whilst only 35% remembered hypoxia, a more common cause with rapid treatment. Secondly, such devices may be subject to study of their effectiveness, albeit with difficulty.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top
• Published on: 29 April 2016

  Appropriate analysis and reporting of cluster randomised trials

  • Steve Goodacre, Senior Lecturer

  Dear Editor

  Dyson et al [1] use a pragmatic design to address an interesting question, but I am concerned that the statistical analysis may be inappropriate and could have led to erroneous conclusions being drawn. The study is a cluster randomised controlled trial. Instead of randomising individual House Officers (HOs), the authors have randomised groups of HOs (those working at the same hospital). This is entirely...

  Show More

  Dear Editor

  Dyson et al [1] use a pragmatic design to address an interesting question, but I am concerned that the statistical analysis may be inappropriate and could have led to erroneous conclusions being drawn. The study is a cluster randomised controlled trial. Instead of randomising individual House Officers (HOs), the authors have randomised groups of HOs (those working at the same hospital). This is entirely appropriate. As the authors point out, randomising individual HOs would risk contamination between the two study groups by HOs sharing aide memoires.

  However, if groups, rather than individuals, are randomised then the use of standard statistical techniques may be inappropriate. These techniques assume that all observations (i.e. all individuals) are independent of each other. Yet in a cluster trial this may not be true. HOs at the same hospital are likely to share characteristics and learning experiences, and thus be more similar to each other than HOs at different hospitals. Assuming independence in these circumstances may lead to an overestimate of statistical power of the study and an underestimate of the P value.

  For this reason, cluster trials should be published with an estimate of the degree of clustering within groups (the intraclass correlation coefficient) and the effect that this has upon statistical power (the design effect). The potential effect of clustering should be considered in the sample size calculation and analysis should take potential clustering into account. The fewer groups randomised and the more individuals there are per group, the greater the potential impact of any clustering. This study involved randomising eight hospitals, with presumably 15-20 HOs per hospital, so the potential effect of clustering should not be ignored.

  Before we can accept the conclusions of this study we need some more information. What was the intraclass correlation coefficient for these data? How many HOs were included from each hospital? Was analysis undertaken at group (hospital) or individual (HO) level? If an individual level analysis was undertaken, was this adjusted for potential clustering?

  Cluster trials are a valuable tool in emergency medicine research, and this study is a good example, yet care needs to be taken in statistical analysis and reporting. This issue has been addressed by the NHS Health Technology Assessment Programme [2], the BMJ [3], and the emergency medicine literature [4]. Guidelines have recently been published for reporting cluster trials [5], we should ensure that articles in the EMJ follow them.

  References

  (1) Dyson E, Voisey S, Hughes S, Higgins B, McQuillan PJ. Educational psychology in medical learning: a randomised controlled trial of two aide memoires for the recall of causes of electromechanical dissociation. Emerg Med J 2004;21:457-460.

  (2) Ukoumunne et al. Methods for evaluating area-wide and organisation- based interventions in health and health care: a systematic review. Health Technology Assessment 1999;3(5).

  (3) Campbell M, Grimshaw J. Cluster randomised trials: time for improvement. BMJ 1998;317:1171-2.

  (4) Wears RL. Statistical methods for analyzing cluster and cluster- randomized data. Academic Emergency Medicine 2002;9:330-341.

  (5) Campbell MK, Elbourne DR, Altman DG, for the CONSORT Group. CONSORT statement: extension to cluster randomised trials. BMJ 2004;328:702-8.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top
• Published on: 29 April 2016

  In need of a PEA?

  • Andrew M. Mason, Immediate Care Physician

  Dear Editor

  For feeble-minded souls such as myself, the aide mémoire has played a key part in my professional life. Some 30 years ago whilst demonstrating anatomy at Cambridge, I devised numerous mnemonics to assist with teaching. To illustrate their power, whilst I have forgotten the names of virtually all my students and most of my fellow demonstrators, I can recall each and every segment of the right lung, all...

  Show More

  Dear Editor

  For feeble-minded souls such as myself, the aide mémoire has played a key part in my professional life. Some 30 years ago whilst demonstrating anatomy at Cambridge, I devised numerous mnemonics to assist with teaching. To illustrate their power, whilst I have forgotten the names of virtually all my students and most of my fellow demonstrators, I can recall each and every segment of the right lung, all the branches of the internal carotid artery and many other obscure anatomical facts without the slightest difficulty, despite their total irrelevance to my current clinical practice. It was with great interest, therefore, that I read the article by Dyson et al [1]. describing their aide mémoire for electromechanical dissociation, and I welcome it as a significant improvement on the traditional ‘4Hs & 4Ts’ method of recalling potentially reversible causes of pulseless electrical activity.

  I was intrigued, however, to observe that the authors had chosen to work with the term ‘electro-mechanical dissociation’ (EMD) rather than ‘pulseless electrical activity’ (PEA) which has become the more commonly used phrase in recent years. I suspect that this may have had something to do with the fact that they were able make the EMD acronym appear in the second of the two triangles (representing Electrolyte + metabolic, Massive hypothermia and Drugs + toxins) as a reminder of the final three causes of EMD. If so, I’m not sure that the word ‘massive’ really works in front of ‘hypothermia’ since it is not an adjective normally associated with this condition, being more commonly applied to describe a heart attack, stroke or pulmonary embolus. For me, use of the word ‘massive’ in this context seems just a bit too contrived.

  After wrestling with the conundrum, I can reveal that the authors could indeed have utilised the more widely accepted PEA acronym, and still have had it appear in the second of the two triangles. This can be done by defining the final three causes of PEA as: Pharmacological + toxic, Electolytic + metabolic, and Algidity. For those unfamiliar with the word algidity, the dictionary definition is chilliness or coldness, and especially (in the medical sense) ‘coldness with collapse’ [2]. An additional advantage of using the PEA rather than the EMD acronym would be that the initial letter of PEA would remind readers that there are 3Ps (Pneumothorax (tension), Pulmonary embolus and Pericardial tamponade) in the first of the two triangles.

  So it has to be two-and-a-half cheers for Dyson et al. and more aide mémoires please!

  References

  (1) Educational psychology in medical learning: a randomised controlled trial of two aide memoires for the recall of causes of electromechanical dissociation. Dyson E, Voisey S, Hughes S, Higgins B, McQuillan PJ. Emerg Med J 2004;21:457-460.

  (2) See: http://cancerweb.ncl.ac.uk/cgi-bin/omd?algidity published by Dept. of Medical Oncology, University of Newcastle upon Tyne.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top