Article Text

Download PDFPDF

PREDICT prioritisation study: establishing the research priorities of paediatric emergency medicine physicians in Australia and New Zealand
  1. Heather Carol Deane1,2,
  2. Catherine L Wilson2,
  3. Franz E Babl1,2,
  4. Stuart R Dalziel3,4,
  5. John Alexander Cheek1,5,
  6. Simon S Craig5,6,
  7. Ed Oakley1,2,
  8. Meredith Borland7,8,
  9. Nicholas G Cheng9,
  10. Michael Zhang10,
  11. Elizabeth Cotterell11,
  12. Tibor Schuster12,13,
  13. David Krieser12,14
  14. on behalf of the PREDICT Research Network
  1. 1 Emergency Department, Royal Children’s Hospital, Melbourne, Victoria, Australia
  2. 2 Emergency Research Group, Murdoch Childrens Research Institute, Parkville, Victoria, Australia
  3. 3 Liggins Institute, University of Auckland, Auckland, New Zealand
  4. 4 Emergency Department, Starship Children’s Hospital, Liggins Institute, University of Auckland, Auckland, New Zealand
  5. 5 Paediatric Emergency Department, Monash Medical Centre, Emergency Program, Monash Health, Melbourne, Victoria, Australia
  6. 6 School of Clinical Sciences, Monash University, Melbourne, Victoria, Australia
  7. 7 Emergency Department, Princess Margaret Hospital for Children, Perth, Western Australia, Australia
  8. 8 Schools of Paediatric and Child Health and Primary, Rural and Aboriginal Healthcare, University of Western Australia, Perth, Western Australia, Australia
  9. 9 Emergency Department, The Children’s Hospital at Westmead, Sydney, New South Wales, Australia
  10. 10 Emergency Department, John Hunter Hospital, Newcastle, New South Wales, Australia
  11. 11 Tablelands Clinical School, University of New England, Armidale, New South Wales, Australia
  12. 12 Department of Paediatrics, University of Melbourne, Melbourne, Victoria, Australia
  13. 13 Clinical Epidemiology and Biostatistics Unit, Melbourne Children’s Trial Centre, Murdoch Childrens Research Institute, Melbourne, Victoria, Australia
  14. 14 Department of Emergency Medicine, Sunshine Hospital, Melbourne, Victoria, Australia
  1. Correspondence to Dr Heather Carol Deane, PREDICT Research Network, Emergency Research Group, Murdoch Childrens Research Institute, Parkville, Melbourne, VIC 3052, Australia; heathercaroldeane{at}gmail.com

Abstract

Background The Paediatric Research in Emergency Departments International Collaborative (PREDICT) performs multicentre research in Australia and New Zealand. Research priorities are difficult to determine, often relying on individual interests or prior work.

Objective To identify the research priorities of paediatric emergency medicine (PEM) specialists working in Australia and New Zealand.

Methods Online surveys were administered in a two-stage, modified Delphi study. Eligible participants were PEM specialists (consultants and senior advanced trainees in PEM from 14 PREDICT sites). Participants submitted up to 3 of their most important research questions (survey 1). Responses were collated and refined, then a shortlist of refined questions was returned to participants for prioritisation (survey 2). A further prioritisation exercise was carried out at a PREDICT meeting using the Hanlon Process of Prioritisation. This determined the priorities of active researchers in PEM including an emphasis on the feasibility of a research question.

Results One hundred and six of 254 (42%) eligible participants responded to survey 1 and 142/245 (58%) to survey 2. One hundred and sixty-eight (66%) took part in either or both surveys. Two hundred forty-six individual research questions were submitted in survey 1. Survey 2 established a prioritised list of 35 research questions. Priority topics from both the Delphi and Hanlon process included high flow oxygenation in intubation, fluid volume resuscitation in sepsis, imaging in cervical spine injury, intravenous therapy for asthma and vasopressor use in sepsis.

Conclusion This prioritisation process has established a list of research questions, which will inform multicentre PEM research in Australia and New Zealand. It has also emphasised the importance of the translation of new knowledge.

  • Paediatric Emergency Medicine
  • Paediatrics
  • Research
  • Methods
View Full Text

Statistics from Altmetric.com

Key messages

What is already known on this subject

  • PREDICT (Paediatric Research in Emergency Departments International Collaborative) is the largest paediatric emergency medicine (PEM) network in Australia and New Zealand.

  • Consensus on research priorities is important for efficient allocation and use of research funds. The Delphi process is proscriptive and is a well-recognised method of reaching consensus.

  • Similar research networks have carried out prioritisation setting exercises to drive their research agenda.

What this study adds

  • This study contains the first published list of 35 PEM research priorities in Australia and New Zealand, providing direction for future research efforts in the field.

  • The key areas include: emergency intubation, asthma management, the management of cervical spine injury and the management of sepsis.

  • The study has also highlighted that greater effort also needs to be directed to the dissemination and translation of new knowledge.

Introduction

Creating evidence in the Emergency Department (ED) can be difficult. Limited research resources need to be carefully allocated and used. It is critical to assess what the research priorities in this area should be according to the people who use the evidence in their daily practice.

The Paediatric Research in Emergency Departments International Collaborative (PREDICT) was formed in 2004, with a vision to improve paediatric emergency medicine (PEM) via the establishment of a sustainable research infrastructure, performing rigorous multicentre research to develop an evidence base.1 The network consists of members from sites across Australia and New Zealand, comprising clinicians from tertiary paediatric EDs, non-tertiary paediatric EDs and mixed adult and paediatric EDs.

To date, PREDICT’s research priorities have evolved from early epidemiological research2 and the research interests of individual members. However, in order to be more inclusive and responsive to the evidence gaps that affect all local practising PEM physicians, the end-users of PREDICT’s research, there is a need to systematically determine the research priority for acute paediatrics in Australia and New Zealand. Such an approach would avoid the inherent bias of just having PREDICT members determine the research agenda, likely increase clinician engagement in future research and, most importantly, produce a research agenda that is relevant to critically ill children presenting to EDs. Other research networks in Western Europe, Australia and North America have conducted similar studies to determine research priorities.3–5

We aimed to identify the research priorities of PEM specialists working in Australia and New Zealand. It is hoped that the PREDICT network can then set a research agenda that will meet its vision.1 We also aimed to identify evidence gaps or a lack of evidence to guide current practice. Secondary aims were to identify any differences in final research priorities between tertiary and non-tertiary centres.

Methods

There are many methods used for reaching consensus; we used two methods in our study: the Delphi method and the Hanlon Process of Prioritisation (HPP).

We conducted a two-stage modified Delphi survey of research priorities with senior medical staff working at 14 PREDICT sites. Delphi studies have been widely used in a number of different settings6 7 and by other research networks for similar prioritisation exercises,3 5 and are well suited to remote surveying. The Delphi method consists of at least two survey rounds. The results of each round are refined by the convener, or a representative group, and reissued to the participants in subsequent rounds, until consensus has been achieved. Our methodology was modelled on similar surveys conducted by the Paediatric Emergency Research United Kingdom and Ireland (PERUKI) network,3 and by the Australian Paediatric Research Network (APRN).5 Ethics approval for the project was sought and granted by The Royal Children’s Hospital, Melbourne (HREC 35035_B).

Surveys were administered via email using a web-based REDCAP (Research Electronic Data Capture) database8 to facilitate survey administration and data management.

Participant selection

We invited senior PEM physicians in Australia and New Zealand to participate. A participant was eligible when they met the following criteria:

  1. a consultant working in PEM in Australia and New Zealand;

  2. a senior advanced trainee in PEM in Australia and New Zealand, defined as a trainee at least 2 years post exam with the Royal Australasian College of Physicians or at least 3 years post primary exam with the Australasian College for Emergency Medicine.

Participants were excluded if on leave during the study period or if they were not treating children as part of their current role. Consent was implied when a participant responded to the survey.

Steering group selection

An expert steering group of 10 PEM physicians was identified from within the PREDICT executive and senior active researchers in PEM. The steering group members were not precluded from taking part in the survey. The two lead authors acted as conveners and were responsible for division of topics in stage 1B, and did not take part in the surveys.

Stage 1A: survey 1

In survey 1 participants were asked, ‘Thinking about your clinical practice in the field of PEM, what are the most important research questions that need addressing?’ Such an open question allows participants to suggest ideas. This is vital at the commencement of a Delphi study.

Participants were invited to submit up to three research questions in the PICO (population, intervention, comparison, outcome) format where possible. Non-PICO questions were later reviewed for their suitability. Reminder emails were sent at regular intervals to those who had not already responded. Survey 1 was open for 6 weeks from April to May 2015.

Participants were also asked to complete baseline demographic data including: year of graduation, level of training, site of practice, work in paediatrics outside of the ED and hours worked per week.

Stage 1B: refinement of research questions

All research questions generated from survey 1 were collated. Eight available members of the steering group independently and rigorously reviewed each question in relation to four criteria:

  1. Is the question a duplicate of another? Duplications were excluded.

  2. Is the question amenable to multicentre research? As PREDICT is a multicentre research network, questions of relevance to just a single centre were excluded.

  3. Is the question amenable to the PICO format? Questions were excluded if >50% of the steering group agreed that the question did not meet both criteria 2 and 3.

  4. Is there current existing evidence to answer the question? A question was excluded if the steering group agreed that existing evidence could answer the question.

Following review by the steering committee first electronically, then via two teleconferences and one face-to-face meeting, research questions were included in the final list of priorities, or excluded from, stage 2.

Stage 2: survey 2

A refined and reduced list of proposed research questions from stage 1B formed survey 2. Participants were asked ‘Thinking about your clinical practice in the field of PEM, how important are the following questions to you in terms of need for future research?’ Respondents were then asked to rate each research question individually on a 7-point Likert scale (1=not a priority, 2=low priority, 3=somewhat priority, 4=neutral, 5=moderate priority, 6=high priority, 7=essential priority).

Quasirandomisation process

In order to prevent bias from survey fatigue, where early questions may be better attended to than later questions, the following quasirandomisation process was used: stage 2 questions were randomly ordered in five different sequences and participants were randomly allocated to complete one of these survey sequences. This was designed as REDCAP did not have a utility to randomise the allocation of questions. Reminder emails were sent at regular intervals to those who had not yet responded. Survey 2 was open for 4 weeks from September to October 2015.

Analysis of Delphi survey 2

Survey 2 results were analysed using StataIC V.13.9 Likert scores to each of the survey 2 questions were merged for analysis with a mean score calculated for each question. Questions were initially prioritised on the basis of their mean score from the Delphi process, consistent with previous similar studies.3

Results were also compared according to the participant’s main site of practice, tertiary versus non-tertiary.

The Hanlon Process of Prioritisation

A further prioritisation process of survey 2 questions was conducted with 21 active PREDICT researchers attending a biannual face-to-face members meeting using the HPP.10 11 The HPP formally weights prevalence, seriousness and feasibility of a given research question. Members independently rated each research question on a scale of 1–10 in relation to the three domains: (A) prevalence (of the condition), (B) seriousness (of the condition), (C) feasibility (of doing the research). Members were asked to consider feasibility of funding, among other issues, when scoring the feasibility domain.

A HPP score was calculated for each question using the mean scores for each of the three domains (A,B,C) as follows: HPP=(A+2B)×C.10 11

Results

Response rates

A total of 168 clinicians responded to our surveys. Sixty-six per cent of eligible participants responded to at least one survey (figure 1).

Figure 1

Delphi survey response rates.

Participants were predominantly PEM specialists (149 (89%)), practising in tertiary EDs (tertiary=144 (86%), non-tertiary=10 (6%), both tertiary and non-tertiary=14 (8%)). Participants predominantly only saw paediatric patients, with 109 (65%) only practising in a paediatric ED, with 51 participants (30%) also seeing adults as part of their current role.

Survey 1

A total of 246 individual research questions were submitted in survey 1. All participants submitted at least one question, 85 participants (80%) submitted two questions and 55 participants (52%) submitted three research questions for consideration.

Stage 1B: refinement of research questions

Survey 1 questions were excluded as follows; 58 duplications, 11 not amenable to Multi-Centre Research (MCR), 24 not amenable to PICO format, 118 pre-existing evidence. Following this process, 35 (14%) questions remained (figure 2).

Figure 2

Refinement process in stage 1A. PICO, population, intervention, control, outcome.

Survey 2: Delphi prioritisation

Mean scores for the 35 research questions in the Delphi survey ranged from 3.58 to 5.37 (rated 1–7) (table 1). The top ranked questions from the Delphi prioritisation related to: high flow nasal oxygenation in emergency intubation, volume of fluid resuscitation in sepsis, role of CT and plain radiography in cervical spine trauma, use of specific intravenous therapy in severe asthma and timing of vasopressor therapy in sepsis.

Table 1

Top 35 PREDICT research priorities (Questions appearing in bold appear top 10 of the Delphi process)

As displayed in figure 3, variation of average question priorities across the five sequences of questions was negligible with differences of less than one priority point for all possible contrasts in distribution quartiles. Overlap of confidence intervals for the median, supports the null hypothesis that assessor rating did not depend on the ordering of questions.

Figure 3

Distribution of average question priorities over five different sequences of question. Boxes represent the IQR, solid lines represent medians and the shaded areas represent 95% CI for the median. Whiskers represent data points that fall within 1.5 IQRs from the lower or upper quartile.

Hanlon Process of Prioritisation

Mean priority scores using the HPP ranged from 47 to 164 (table 1). The top ranked priorities from Hanlon were choice of intravenous asthma agents, oral versus intravenous therapy for urinary tract infections, respiratory support in asthma, cervical spine decision rules and the use of a preintubation checklist.

Differences in research priorities according to participant’s main site of practice

Participants from non-tertiary sites ranked the management of acute limp, the management of greenstick fractures and the management of the febrile neonate more highly than participants from tertiary sites.

Discussion

This study has identified multicentre research network priorities in PEM and captured the ideas and opinions of senior PEM specialists across Australia and New Zealand. The presented list of 35 research questions will guide research efforts within PREDICT over the coming years and assist in underpinning funding requests for future projects. Top ranking areas included asthma management, emergency intubation, imaging of suspected cervical spine injuries and the management of sepsis.

Asthma is one of the most common reasons children seek care in EDs.2 Previous work within the PREDICT network identified the high variability in practice in the treatment of acute severe paediatric asthma,12 and this has been confirmed by more recent work in the UK and Ireland.13 Severe acute paediatric asthma is relatively rare (around 3% of paediatric ED asthma attendances), and there is great practice variability with many choices of intravenous therapy (including ketamine, magnesium, aminophylline and salbutamol) and a weak evidence base.

Two questions regarding emergency intubation featured in the top 10 in both priority lists.

One advantage of network research is an ability to investigate diseases and procedures that may be rare in a single ED, pooling the experience of multiple sites.1 14 Emergent airway management is one such procedure and recognition of the utility of multicentre research in studying events with such low prevalence may have been recognised by the respondents.15

The management of cervical spine injuries in the paediatric population is a challenging area lacking high-level evidence; in addition, significant injury is rare. Three questions regarding cervical spine injury featured in the final list of priorities. The existing cervical spine rules in adults have not been validated in the paediatric population, making it difficult to decide on appropriate management.16 17 Significant issues specific to children include concerns with exposure to radiation, the difficulty of patient compliance with immobilisation and imaging, and the choice of imaging modality.

Five questions regarding the management of sepsis were included in the final 35 priorities. The focus for these questions was largely around the volume of fluid, the type of fluid and the use of vasopressors. This likely reflects consensus guidelines based on limited evidence and evolving conflicting evidence such as the findings from the FEAST trial.18–20

Mean priority scores using the Delphi method had a narrow range, indicating that the priorities were closely ranked. Additionally, the use of the HPP (completed with a smaller group) adds complementary perspectives of feasibility, seriousness and prevalence. This approach allowed us to consider whether our final list included achievable research questions of importance to the population.

Of note, nearly half of all originally submitted questions were deemed to have sufficient existing evidence to answer them. Some of the broad topic areas included the use of analgesia and sedation, fluid and circulatory management choices in the unwell child, use of biochemical markers in sepsis, steroid use in respiratory presentations and management strategies in bronchiolitis. This suggests that considerable gaps remain between the published literature and individual clinicians’ knowledge, consistent with previous research suggesting a lag of up to 17 years for research findings to reach clinical practice.21 Furthermore, this finding underscores the need to conduct appropriate systematic literature reviews prior to commencing research in a specific topic—an important measure which will reduce research waste. Finally, these questions provide an opportunity to focus further knowledge translation strategies within PEM.

Knowledge translation is an area of great importance and it is the responsibility of all organisations, training bodies and research networks involved in clinical research. This study has highlighted the need for the network to focus almost equal effort and funds towards the translation of new knowledge into clinical practice in the acute care setting. Studies of knowledge translation strategies focusing on changing practice for paediatric patients within local EDs and general paediatric wards are currently under way to help inform local practice.22 23

The PREDICT network has been operational for over 10 years and in 2014 was awarded an NHMRC grant as a Centre for Research Excellence in Paediatric Emergency Medicine (ID 1058560). This has supported the network to undertake strategic planning to map out future directions and areas for development. Since the identification of the most important research priorities in our field we have focused research efforts towards seven of the top 10 research questions highlighted in this prioritisation. As this prioritisation was obtained in a national context funding agencies should use the findings to inform decisions regarding individual research proposals.

The completion of our prioritisation process marks the completion of prioritisation exercises by three of the large international PEM networks.3 4 Similarities between our study and those conducted in the USA and UK3 4 include: asthma treatment, clinical decision making in possible cervical spine injury, the use of tranexamic acid in blunt trauma and reduction of medical errors.

Some areas included in other prioritisation exercises and not included in our final list included: injury prevention, mental health, acuity scales4, clinical decision rules in children with petechiae, the cost effectiveness of observation wards and head injury rules.3

Limitations

Our response rate is similar to other Delphi studies3 5 (see table 2) and we did not capture the opinions of 34% (n=86) of invited participants in either survey. In the age of high-volume electronic communication, participant fatigue and survey burden are factors which affect response rates.24 However, in a study where we want to capture a wide range of opinions and reach consensus, this response rate is a potential limitation.

Table 2

Response rate compared with other Delphi studies

The Delphi method is not prescriptive, and therefore no single standardised method exists. It can essentially be tailored to suit the needs of the particular group and study and this creates potential for bias. However, we have deliberately based our methodology on the PERUKI study to allow direct comparison of results.3

The participants in this study were from 14 PREDICT sites in Australia and New Zealand. As such, the results are biased towards tertiary paediatric EDs. However, the 14 sites included five mixed paediatric/adult EDs. Furthermore, a number of the staff in the tertiary paediatric EDs also hold clinical positions in mixed paediatric/adult EDs. Thus, the findings are likely to be broadly generalisable to paediatric acute care presentations in Australia and New Zealand.

It is also worth noting that this study includes only the opinions and responses of mainly non-academic PEM specialists. We have not included the perspectives of ED nurses, allied health physicians, patients, parents or caregivers. Other prioritisation projects, such as those conducted by the James Lind Alliance in the UK, have formed partnerships incorporating wide clinical and community engagement. Their recently published Emergency Medicine Research Priority Setting Partnership25 established 10 research priorities for emergency medicine in the UK. Of note, five of these priorities focused on broad questions regarding delivery of care (eg, ED crowding, optimising care of mental health patients, appropriateness of ED for care for certain patient groups), two on staffing priorities, with only three focusing on specific clinical care of patients. There were no priorities directly of relevance to paediatric patients, a group often overlooked in general medical research. These differences in priorities to those found in our study may reflect the difference between a paediatric and adult-focused study, or reflect the difference in perspectives and potential importance of incorporating a broader range of participants (including clinicians from other clinical disciplines, patients and carers in particular). Our final list of priorities remains clinically focused.

Conclusions

This study has identified the research priorities of PEM specialists across Australia and New Zealand. The outcome of our modified Delphi process, together with the HPP, is a defined set of clinical research priorities to inform future multicentre PEM research across Australia and New Zealand. Key research areas prioritised include emergency intubation, asthma management, the management of cervical spine injury and the management of sepsis.3 4 Greater research effort to translate established evidence is also indicated.

Acknowledgments

We thank the participants for responding to the surveys. We thank Dr Harriet Hiscock and Ms Alisha Gulenc from APRN and Dr Stuart Hartshorn from PERUKI for their advice and support in the planning stages of our study.

References

View Abstract

Footnotes

  • Contributors All authors contributed to the final manuscript, including conception and design of the study, drafting of the manuscript and final approval of the manuscript.

  • Funding The study was funded by grants from the National Health and Medical Research Council (Centre of Research Excellence for Paediatric Emergency Medicine GNT1058560), Canberra, Australia; the Murdoch Childrens Research Institute, Melbourne, Australia; and supported by the Victorian Government’s Infrastructure Support Program, Melbourne, Australia. FEB’s time was in part funded by a grant from the Royal Children’s Hospital Foundation, Melbourne, Australia. SRD’s time was in part funded by the Health Research Council of New Zealand (HRC13/556).

  • Competing interests None declared.

  • Ethics approval Ethics approval for the project was sought andgranted by The Royal Children’s Hospital, Melbourne (HREC35035_B).

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.