You are here

Article Text

Article menu
Download PDFPDF
Original research
Inter-rater reliability in a bespoke scoring tool: the Paediatric Observation Priority Score
  1. Adam Bonfield1,
  2. Damian Roland2,3
  1. 1 College of Medicine Biological Sciences and Psychology, University of Leicester, Leicester, UK
  2. 2 SAPPHIRE Group, Health Sciences, University of Leicester, Leicester, UK
  3. 3 Paediatric Emergency Medicine Leicester Academic (PEMLA) Group, Leicester Hospitals, Leicester, UK
  1. Correspondence to Dr Adam Bonfield, College of Medicine, Biological Sciences and Psychology, University of Leicester College of Medicine Biological Sciences and Psychology, Leicester LE1 7HA, UK; ab798{at}le.ac.uk

Abstract

Objective Inter-rater reliability (IRR) is rarely determined for scoring systems used to recognise deterioration in children. Thus, the primary objective of this study was to determine the IRR of the Paediatric Observation Priority Score (POPS), a bespoke paediatric scoring system for ED use. The IRR of both the overall POPS and its individual parameters are to be investigated.

Methods This is an experimental, single-centre study based in the Leicester children’s ED, England. A purposive sample of nursing, healthcare assistants and doctors were recruited from October 2017 to December 2017. Eleven prerecorded video assessments were shown to 45 participants asked to generate a POPS for each child. The participants were blinded to each other’s scores and the triage POPS. IRR among the participants POPS was measured using the intraclass correlation coefficient (ICC). Fleiss kappa was used to determine the IRR of individual parameters.

Results Overall, the ICC across all assessments by staff was 0.84 (95% CI 0.71 to 0.95). Fleiss kappa for parameters ranged from 0.87 to 1 for oxygen saturation, 0.48 to 0.91 for work of breathing, 0.55 to 1 for response, 0.46 to 0.87 for gut feeling, 0.53 to 1 for medical history, 0.76 to 1 for heart rate, 0.44 to 0.96 for respiratory rate and 0.51 to 1 for temperature.

Conclusions This study suggests there is statistically moderate to excellent IRR of the POPS when assessing a variety of clinical presentations between different healthcare professionals with a range of experience.

  • paediatrics
  • clinical assessment, effectiveness
  • performance improvement
  • risk management
  • triage

https://doi.org/10.1136/emermed-2018-208268

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Twitter @TeachMePaeds, @damian_roland

    • Contributors All named authors contributed to the development and process of this research project. AB designed the protocol, consented participants, collected data, analysed data and wrote the manuscript. DR supervised in all of the above areas and contributed to the editorial process of the manuscript for publication.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests AB is a National Institute for Health Research Academic Clinical Fellow.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.