PREDICT prioritisation study: establishing the research priorities of paediatric emergency medicine physicians in Australia and New Zealand
===========================================================================================================================================

* Heather Carol Deane
* Catherine L Wilson
* Franz E Babl
* Stuart R Dalziel
* John Alexander Cheek
* Simon S Craig
* Ed Oakley
* Meredith Borland
* Nicholas G Cheng
* Michael Zhang
* Elizabeth Cotterell
* Tibor Schuster
* David Krieser

## Abstract

**Background** The Paediatric Research in Emergency Departments International Collaborative (PREDICT) performs multicentre research in Australia and New Zealand. Research priorities are difficult to determine, often relying on individual interests or prior work.

**Objective** To identify the research priorities of paediatric emergency medicine (PEM) specialists working in Australia and New Zealand.

**Methods** Online surveys were administered in a two-stage, modified Delphi study. Eligible participants were PEM specialists (consultants and senior advanced trainees in PEM from 14 PREDICT sites). Participants submitted up to 3 of their most important research questions (survey 1). Responses were collated and refined, then a shortlist of refined questions was returned to participants for prioritisation (survey 2). A further prioritisation exercise was carried out at a PREDICT meeting using the Hanlon Process of Prioritisation. This determined the priorities of active researchers in PEM including an emphasis on the feasibility of a research question.

**Results** One hundred and six of 254 (42%) eligible participants responded to survey 1 and 142/245 (58%) to survey 2. One hundred and sixty-eight (66%) took part in either or both surveys. Two hundred forty-six individual research questions were submitted in survey 1. Survey 2 established a prioritised list of 35 research questions. Priority topics from both the Delphi and Hanlon process included high flow oxygenation in intubation, fluid volume resuscitation in sepsis, imaging in cervical spine injury, intravenous therapy for asthma and vasopressor use in sepsis.

**Conclusion** This prioritisation process has established a list of research questions, which will inform multicentre PEM research in Australia and New Zealand. It has also emphasised the importance of the translation of new knowledge.

*   Paediatric Emergency Medicine
*   Paediatrics
*   Research
*   Methods

### Key messages

#### What is already known on this subject?

*   PREDICT (Paediatric Research in Emergency Departments International Collaborative) is the largest paediatric emergency medicine (PEM) network in Australia and New Zealand.

*   Consensus on research priorities is important for efficient allocation and use of research funds. The Delphi process is proscriptive and is a well-recognised method of reaching consensus.

*   Similar research networks have carried out prioritisation setting exercises to drive their research agenda.

#### What this study adds

*   This study contains the first published list of 35 PEM research priorities in Australia and New Zealand, providing direction for future research efforts in the field.

*   The key areas include: emergency intubation, asthma management, the management of cervical spine injury and the management of sepsis.

*   The study has also highlighted that greater effort also needs to be directed to the dissemination and translation of new knowledge.

## Introduction

Creating evidence in the Emergency Department (ED) can be difficult. Limited research resources need to be carefully allocated and used. It is critical to assess what the research priorities in this area should be according to the people who use the evidence in their daily practice.

The Paediatric Research in Emergency Departments International Collaborative (PREDICT) was formed in 2004, with a vision to improve paediatric emergency medicine (PEM) via the establishment of a sustainable research infrastructure, performing rigorous multicentre research to develop an evidence base.1 The network consists of members from sites across Australia and New Zealand, comprising clinicians from tertiary paediatric EDs, non-tertiary paediatric EDs and mixed adult and paediatric EDs.

To date, PREDICT’s research priorities have evolved from early epidemiological research2 and the research interests of individual members. However, in order to be more inclusive and responsive to the evidence gaps that affect all local practising PEM physicians, the end-users of PREDICT’s research, there is a need to systematically determine the research priority for acute paediatrics in Australia and New Zealand. Such an approach would avoid the inherent bias of just having PREDICT members determine the research agenda, likely increase clinician engagement in future research and, most importantly, produce a research agenda that is relevant to critically ill children presenting to EDs. Other research networks in Western Europe, Australia and North America have conducted similar studies to determine research priorities.3–5

We aimed to identify the research priorities of PEM specialists working in Australia and New Zealand. It is hoped that the PREDICT network can then set a research agenda that will meet its vision.1 We also aimed to identify evidence gaps or a lack of evidence to guide current practice. Secondary aims were to identify any differences in final research priorities between tertiary and non-tertiary centres.

## Methods

There are many methods used for reaching consensus; we used two methods in our study: the Delphi method and the Hanlon Process of Prioritisation (HPP).

We conducted a two-stage modified Delphi survey of research priorities with senior medical staff working at 14 PREDICT sites. Delphi studies have been widely used in a number of different settings6 7 and by other research networks for similar prioritisation exercises,3 5 and are well suited to remote surveying. The Delphi method consists of at least two survey rounds. The results of each round are refined by the convener, or a representative group, and reissued to the participants in subsequent rounds, until consensus has been achieved. Our methodology was modelled on similar surveys conducted by the Paediatric Emergency Research United Kingdom and Ireland (PERUKI) network,3 and by the Australian Paediatric Research Network (APRN).5 Ethics approval for the project was sought and granted by The Royal Children’s Hospital, Melbourne (HREC 35035_B).

Surveys were administered via email using a web-based REDCAP (Research Electronic Data Capture) database8 to facilitate survey administration and data management.

### Participant selection

We invited senior PEM physicians in Australia and New Zealand to participate. A participant was eligible when they met the following criteria:

1.  a consultant working in PEM in Australia and New Zealand;

2.  a senior advanced trainee in PEM in Australia and New Zealand, defined as a trainee at least 2 years post exam with the Royal Australasian College of Physicians or at least 3 years post primary exam with the Australasian College for Emergency Medicine.

Participants were excluded if on leave during the study period or if they were not treating children as part of their current role. Consent was implied when a participant responded to the survey.

### Steering group selection

An expert steering group of 10 PEM physicians was identified from within the PREDICT executive and senior active researchers in PEM. The steering group members were not precluded from taking part in the survey. The two lead authors acted as conveners and were responsible for division of topics in stage 1B, and did not take part in the surveys.

#### Stage 1A: survey 1

In survey 1 participants were asked, *‘Thinking about your clinical practice in the field of PEM, what are the most important research questions that need addressing?’* Such an open question allows participants to suggest ideas. This is vital at the commencement of a Delphi study.

Participants were invited to submit up to three research questions in the PICO (population, intervention, comparison, outcome) format where possible. Non-PICO questions were later reviewed for their suitability. Reminder emails were sent at regular intervals to those who had not already responded. Survey 1 was open for 6 weeks from April to May 2015.

Participants were also asked to complete baseline demographic data including: year of graduation, level of training, site of practice, work in paediatrics outside of the ED and hours worked per week.

#### Stage 1B: refinement of research questions

All research questions generated from survey 1 were collated. Eight available members of the steering group independently and rigorously reviewed each question in relation to four criteria:

1.  Is the question a duplicate of another? Duplications were excluded.

2.  Is the question amenable to multicentre research? As PREDICT is a multicentre research network, questions of relevance to just a single centre were excluded.

3.  Is the question amenable to the PICO format? Questions were excluded if >50% of the steering group agreed that the question did not meet both criteria 2 and 3.

4.  Is there current existing evidence to answer the question? A question was excluded if the steering group agreed that existing evidence could answer the question.

Following review by the steering committee first electronically, then via two teleconferences and one face-to-face meeting, research questions were included in the final list of priorities, or excluded from, stage 2.

#### Stage 2: survey 2

A refined and reduced list of proposed research questions from stage 1B formed survey 2. Participants were asked ‘*Thinking about your clinical practice in the field of PEM, how important are the following questions to you in terms of need for future research?’* Respondents were then asked to rate each research question individually on a 7-point Likert scale (1=not a priority, 2=low priority, 3=somewhat priority, 4=neutral, 5=moderate priority, 6=high priority, 7=essential priority).

#### Quasirandomisation process

In order to prevent bias from survey fatigue, where early questions may be better attended to than later questions, the following quasirandomisation process was used: stage 2 questions were randomly ordered in five different sequences and participants were randomly allocated to complete one of these survey sequences. This was designed as REDCAP did not have a utility to randomise the allocation of questions. Reminder emails were sent at regular intervals to those who had not yet responded. Survey 2 was open for 4 weeks from September to October 2015.

### Analysis of Delphi survey 2

Survey 2 results were analysed using StataIC V.13.9 Likert scores to each of the survey 2 questions were merged for analysis with a mean score calculated for each question. Questions were initially prioritised on the basis of their mean score from the Delphi process, consistent with previous similar studies.3

Results were also compared according to the participant’s main site of practice, tertiary versus non-tertiary.

#### The Hanlon Process of Prioritisation

A further prioritisation process of survey 2 questions was conducted with 21 active PREDICT researchers attending a biannual face-to-face members meeting using the HPP.10 11 The HPP formally weights prevalence, seriousness and feasibility of a given research question. Members independently rated each research question on a scale of 1–10 in relation to the three domains: (A) prevalence (of the condition), (B) seriousness (of the condition), (C) feasibility (of doing the research). Members were asked to consider feasibility of funding, among other issues, when scoring the feasibility domain.

A HPP score was calculated for each question using the mean scores for each of the three domains (A,B,C) as follows: HPP=(A+2B)×C.10 11

## Results

### Response rates

A total of 168 clinicians responded to our surveys. Sixty-six per cent of eligible participants responded to at least one survey (figure 1).

![Figure 1](http://emj.bmj.com/https://emj.bmj.com/content/emermed/early/2017/08/30/emermed-2017-206727/F1.medium.gif)

[Figure 1](http://emj.bmj.com/content/early/2017/08/30/emermed-2017-206727/F1)

Figure 1 
Delphi survey response rates.

Participants were predominantly PEM specialists (149 (89%)), practising in tertiary EDs (tertiary=144 (86%), non-tertiary=10 (6%), both tertiary and non-tertiary=14 (8%)). Participants predominantly only saw paediatric patients, with 109 (65%) only practising in a paediatric ED, with 51 participants (30%) also seeing adults as part of their current role.

### Survey 1

A total of 246 individual research questions were submitted in survey 1. All participants submitted at least one question, 85 participants (80%) submitted two questions and 55 participants (52%) submitted three research questions for consideration.

### Stage 1B: refinement of research questions

Survey 1 questions were excluded as follows; 58 duplications, 11 not amenable to Multi-Centre Research (MCR), 24 not amenable to PICO format, 118 pre-existing evidence. Following this process, 35 (14%) questions remained (figure 2).

![Figure 2](http://emj.bmj.com/https://emj.bmj.com/content/emermed/early/2017/08/30/emermed-2017-206727/F2.medium.gif)

[Figure 2](http://emj.bmj.com/content/early/2017/08/30/emermed-2017-206727/F2)

Figure 2 
Refinement process in stage 1A. PICO, population, intervention, control, outcome.

### Survey 2: Delphi prioritisation

Mean scores for the 35 research questions in the Delphi survey ranged from 3.58 to 5.37 (rated 1–7) (table 1). The top ranked questions from the Delphi prioritisation related to: high flow nasal oxygenation in emergency intubation, volume of fluid resuscitation in sepsis, role of CT and plain radiography in cervical spine trauma, use of specific intravenous therapy in severe asthma and timing of vasopressor therapy in sepsis.

View this table:
[Table 1](http://emj.bmj.com/content/early/2017/08/30/emermed-2017-206727/T1)

Table 1 
Top 35 PREDICT research priorities (Questions appearing in bold appear top 10 of the Delphi process)

As displayed in figure 3, variation of average question priorities across the five sequences of questions was negligible with differences of less than one priority point for all possible contrasts in distribution quartiles. Overlap of confidence intervals for the median, supports the null hypothesis that assessor rating did not depend on the ordering of questions.

![Figure 3](http://emj.bmj.com/https://emj.bmj.com/content/emermed/early/2017/08/30/emermed-2017-206727/F3.medium.gif)

[Figure 3](http://emj.bmj.com/content/early/2017/08/30/emermed-2017-206727/F3)

Figure 3 
Distribution of average question priorities over five different sequences of question. Boxes represent the IQR, solid lines represent medians and the shaded areas represent 95% CI for the median. Whiskers represent data points that fall within 1.5 IQRs from the lower or upper quartile.

### Hanlon Process of Prioritisation

Mean priority scores using the HPP ranged from 47 to 164 (table 1). The top ranked priorities from Hanlon were choice of intravenous asthma agents, oral versus intravenous therapy for urinary tract infections, respiratory support in asthma, cervical spine decision rules and the use of a preintubation checklist.

### Differences in research priorities according to participant’s main site of practice

Participants from non-tertiary sites ranked the management of acute limp, the management of greenstick fractures and the management of the febrile neonate more highly than participants from tertiary sites.

## Discussion

This study has identified multicentre research network priorities in PEM and captured the ideas and opinions of senior PEM specialists across Australia and New Zealand. The presented list of 35 research questions will guide research efforts within PREDICT over the coming years and assist in underpinning funding requests for future projects. Top ranking areas included asthma management, emergency intubation, imaging of suspected cervical spine injuries and the management of sepsis.

Asthma is one of the most common reasons children seek care in EDs.2 Previous work within the PREDICT network identified the high variability in practice in the treatment of acute severe paediatric asthma,12 and this has been confirmed by more recent work in the UK and Ireland.13 Severe acute paediatric asthma is relatively rare (around 3% of paediatric ED asthma attendances), and there is great practice variability with many choices of intravenous therapy (including ketamine, magnesium, aminophylline and salbutamol) and a weak evidence base.

Two questions regarding emergency intubation featured in the top 10 in both priority lists.

One advantage of network research is an ability to investigate diseases and procedures that may be rare in a single ED, pooling the experience of multiple sites.1 14 Emergent airway management is one such procedure and recognition of the utility of multicentre research in studying events with such low prevalence may have been recognised by the respondents.15

The management of cervical spine injuries in the paediatric population is a challenging area lacking high-level evidence; in addition, significant injury is rare. Three questions regarding cervical spine injury featured in the final list of priorities. The existing cervical spine rules in adults have not been validated in the paediatric population, making it difficult to decide on appropriate management.16 17 Significant issues specific to children include concerns with exposure to radiation, the difficulty of patient compliance with immobilisation and imaging, and the choice of imaging modality.

Five questions regarding the management of sepsis were included in the final 35 priorities. The focus for these questions was largely around the volume of fluid, the type of fluid and the use of vasopressors. This likely reflects consensus guidelines based on limited evidence and evolving conflicting evidence such as the findings from the FEAST trial.18–20

Mean priority scores using the Delphi method had a narrow range, indicating that the priorities were closely ranked. Additionally, the use of the HPP (completed with a smaller group) adds complementary perspectives of feasibility, seriousness and prevalence. This approach allowed us to consider whether our final list included achievable research questions of importance to the population.

Of note, nearly half of all originally submitted questions were deemed to have sufficient existing evidence to answer them. Some of the broad topic areas included the use of analgesia and sedation, fluid and circulatory management choices in the unwell child, use of biochemical markers in sepsis, steroid use in respiratory presentations and management strategies in bronchiolitis. This suggests that considerable gaps remain between the published literature and individual clinicians’ knowledge, consistent with previous research suggesting a lag of up to 17 years for research findings to reach clinical practice.21 Furthermore, this finding underscores the need to conduct appropriate systematic literature reviews prior to commencing research in a specific topic—an important measure which will reduce research waste. Finally, these questions provide an opportunity to focus further knowledge translation strategies within PEM.

Knowledge translation is an area of great importance and it is the responsibility of all organisations, training bodies and research networks involved in clinical research. This study has highlighted the need for the network to focus almost equal effort and funds towards the translation of new knowledge into clinical practice in the acute care setting. Studies of knowledge translation strategies focusing on changing practice for paediatric patients within local EDs and general paediatric wards are currently under way to help inform local practice.22 23

The PREDICT network has been operational for over 10 years and in 2014 was awarded an NHMRC grant as a Centre for Research Excellence in Paediatric Emergency Medicine (ID *1058560)*. This has supported the network to undertake strategic planning to map out future directions and areas for development. Since the identification of the most important research priorities in our field we have focused research efforts towards seven of the top 10 research questions highlighted in this prioritisation. As this prioritisation was obtained in a national context funding agencies should use the findings to inform decisions regarding individual research proposals.

The completion of our prioritisation process marks the completion of prioritisation exercises by three of the large international PEM networks.3 4 Similarities between our study and those conducted in the USA and UK3 4 include: asthma treatment, clinical decision making in possible cervical spine injury, the use of tranexamic acid in blunt trauma and reduction of medical errors.

Some areas included in other prioritisation exercises and not included in our final list included: injury prevention, mental health, acuity scales4 ,clinical decision rules in children with petechiae, the cost effectiveness of observation wards and head injury rules.3

## Limitations

Our response rate is similar to other Delphi studies3 5 (see table 2) and we did not capture the opinions of 34% (n=86) of invited participants in either survey. In the age of high-volume electronic communication, participant fatigue and survey burden are factors which affect response rates.24 However, in a study where we want to capture a wide range of opinions and reach consensus, this response rate is a potential limitation.

View this table:
[Table 2](http://emj.bmj.com/content/early/2017/08/30/emermed-2017-206727/T2)

Table 2 
Response rate compared with other Delphi studies

The Delphi method is not prescriptive, and therefore no single standardised method exists. It can essentially be tailored to suit the needs of the particular group and study and this creates potential for bias. However, we have deliberately based our methodology on the PERUKI study to allow direct comparison of results.3

The participants in this study were from 14 PREDICT sites in Australia and New Zealand. As such, the results are biased towards tertiary paediatric EDs. However, the 14 sites included five mixed paediatric/adult EDs. Furthermore, a number of the staff in the tertiary paediatric EDs also hold clinical positions in mixed paediatric/adult EDs. Thus, the findings are likely to be broadly generalisable to paediatric acute care presentations in Australia and New Zealand.

It is also worth noting that this study includes only the opinions and responses of mainly non-academic PEM specialists. We have not included the perspectives of ED nurses, allied health physicians, patients, parents or caregivers. Other prioritisation projects, such as those conducted by the James Lind Alliance in the UK, have formed partnerships incorporating wide clinical and community engagement. Their recently published Emergency Medicine Research Priority Setting Partnership25 established 10 research priorities for emergency medicine in the UK. Of note, five of these priorities focused on broad questions regarding delivery of care (eg, ED crowding, optimising care of mental health patients, appropriateness of ED for care for certain patient groups), two on staffing priorities, with only three focusing on specific clinical care of patients. There were no priorities directly of relevance to paediatric patients, a group often overlooked in general medical research. These differences in priorities to those found in our study may reflect the difference between a paediatric and adult-focused study, or reflect the difference in perspectives and potential importance of incorporating a broader range of participants (including clinicians from other clinical disciplines, patients and carers in particular). Our final list of priorities remains clinically focused.

## Conclusions

This study has identified the research priorities of PEM specialists across Australia and New Zealand. The outcome of our modified Delphi process, together with the HPP, is a defined set of clinical research priorities to inform future multicentre PEM research across Australia and New Zealand. Key research areas prioritised include emergency intubation, asthma management, the management of cervical spine injury and the management of sepsis.3 4 Greater research effort to translate established evidence is also indicated.

## Acknowledgments

We thank the participants for responding to the surveys. We thank Dr Harriet Hiscock and Ms Alisha Gulenc from APRN and Dr Stuart Hartshorn from PERUKI for their advice and support in the planning stages of our study.

## Footnotes

*   Contributors All authors contributed to the final manuscript, including conception and design of the study, drafting of the manuscript and final approval of the manuscript.

*   Funding The study was funded by grants from the National Health and Medical Research Council (Centre of Research Excellence for Paediatric Emergency Medicine GNT1058560), Canberra, Australia; the Murdoch Childrens Research Institute, Melbourne, Australia; and supported by the Victorian Government’s Infrastructure Support Program, Melbourne, Australia. FEB’s time was in part funded by a grant from the Royal Children’s Hospital Foundation, Melbourne, Australia. SRD’s time was in part funded by the Health Research Council of New Zealand (HRC13/556).

*   Competing interests None declared.

*   Ethics approval Ethics approval for the project was sought andgranted by The Royal Children’s Hospital, Melbourne (HREC35035_B).

*   Provenance and peer review Not commissioned; externally peer reviewed.

## References

1.  Babl F, Borland M, Ngo P, et al. Paediatric Research in Emergency Departments International Collaborative (PREDICT): first steps towards the development of an Australian and New Zealand research network. Emerg Med Australas 2006;18:143–7.[doi:10.1111/j.1742-6723.2006.00823.x](http://dx.doi.org/10.1111/j.1742-6723.2006.00823.x)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1111/j.1742-6723.2006.00823.x&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=16669940&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 

2.  Acworth J, Babl F, Borland M, et al. Patterns of presentation to the Australian and New Zealand Paediatric Emergency Research Network. Emerg Med Australas 2009;21:59–66.[doi:10.1111/j.1742-6723.2009.01154.x](http://dx.doi.org/10.1111/j.1742-6723.2009.01154.x)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1111/j.1742-6723.2009.01154.x&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=19254314&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=000263830900009&link_type=ISI) 

3.  Hartshorn S, O'Sullivan R, Maconochie IK, et al. Establishing the research priorities of paediatric emergency medicine clinicians in the UK and Ireland. Emerg Med J 2015;32:864–8.[doi:10.1136/emermed-2014-204484](http://dx.doi.org/10.1136/emermed-2014-204484)
    
    [Abstract/FREE Full Text](http://emj.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo5OiIzMi8xMS84NjQiO3M6NDoiYXRvbSI7czo1MDoiL2VtZXJtZWQvZWFybHkvMjAxNy8wOC8zMC9lbWVybWVkLTIwMTctMjA2NzI3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  Miller SZ, Rincón H, Kuppermann N. Pediatric Emergency Care Applied Research Network. Revisiting the emergency medicine services for children research agenda: priorities for multicenter research in pediatric emergency care. Acad Emerg Med 2008;15:377–83.[doi:10.1111/j.1553-2712.2008.00072.x](http://dx.doi.org/10.1111/j.1553-2712.2008.00072.x)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1111/j.1553-2712.2008.00072.x&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=18370994&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=000254412600012&link_type=ISI) 

5.  Rudolph S, Hiscock H, Price A, et al. What research questions matter to Australian paediatricians? National Delphi Study. J Paediatr Child Health 2009;45:704–10.[doi:10.1111/j.1440-1754.2009.01598.x](http://dx.doi.org/10.1111/j.1440-1754.2009.01598.x)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1111/j.1440-1754.2009.01598.x&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=19863713&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=000273005200004&link_type=ISI) 

6.  Haeusler GM, Phillips RS, Lehrnbecher T, et al. Core outcomes and definitions for pediatric fever and neutropenia research: a consensus statement from an international panel. Pediatr Blood Cancer 2015;62:483–9.[doi:10.1002/pbc.25335](http://dx.doi.org/10.1002/pbc.25335)
    
    

7.  Mackway-Jones K, Carley SD, Robson J. Planning for major incidents involving children by implementing a Delphi study. Arch Dis Child 1999;80:410–3.[doi:10.1136/adc.80.5.410](http://dx.doi.org/10.1136/adc.80.5.410)
    
    [Abstract/FREE Full Text](http://emj.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo4OiI4MC81LzQxMCI7czo0OiJhdG9tIjtzOjUwOiIvZW1lcm1lZC9lYXJseS8yMDE3LzA4LzMwL2VtZXJtZWQtMjAxNy0yMDY3MjcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

8.  Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81.[doi:10.1016/j.jbi.2008.08.010](http://dx.doi.org/10.1016/j.jbi.2008.08.010)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1016/j.jbi.2008.08.010&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=18929686&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=000264958800018&link_type=ISI) 

9.  Stata Statistical Software. Relase 13 ed. College Station TX: StataCorp LP, 2013.
    
    

10. NACCHO. First things First: prioritizing Health problems, 2013. [http://www.countyhealthrankings.org/resources/first-things-first-prioritizing-health-problems](http://www.countyhealthrankings.org/resources/first-things-first-prioritizing-health-problems).
    
    

11.  Assessment Protocol for Excellence in Public Health (APEXPH) . Prioritization. [https://www.cdc.gov/nphpsp/documents/prioritization-section-from-apexph-in-practice.pdf](https://www.cdc.gov/nphpsp/documents/prioritization-section-from-apexph-in-practice.pdf).
    
    

12. Babl FE, Sheriff N, Borland M, et al. Paediatric acute asthma management in Australia and New Zealand: practice patterns in the context of clinical practice guidelines. Arch Dis Child 2008;93:307–12.[doi:10.1136/adc.2007.125062](http://dx.doi.org/10.1136/adc.2007.125062)
    
    [Abstract/FREE Full Text](http://emj.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo4OiI5My80LzMwNyI7czo0OiJhdG9tIjtzOjUwOiIvZW1lcm1lZC9lYXJseS8yMDE3LzA4LzMwL2VtZXJtZWQtMjAxNy0yMDY3MjcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

13. Morris I, Lyttle MD, O'Sullivan R, et al. Which intravenous bronchodilators are being administered to children presenting with acute severe wheeze in the UK and Ireland? Thorax 2015;70:88–91.[doi:10.1136/thoraxjnl-2014-206041](http://dx.doi.org/10.1136/thoraxjnl-2014-206041)
    
    [Abstract/FREE Full Text](http://emj.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjc6IjcwLzEvODgiO3M6NDoiYXRvbSI7czo1MDoiL2VtZXJtZWQvZWFybHkvMjAxNy8wOC8zMC9lbWVybWVkLTIwMTctMjA2NzI3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

14. Pediatric Emergency Care Applied Research Network. The Pediatric Emergency Care Applied Research Network (PECARN): rationale, development, and first steps. Pediatr Emerg Care 2003;19:185–93.[doi:10.1097/01.pec.0000081245.98249.6e](http://dx.doi.org/10.1097/01.pec.0000081245.98249.6e)
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=12813308&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=000183594400013&link_type=ISI) 

15. Long E, Sabato S, Babl FE. Endotracheal intubation in the pediatric emergency department. Paediatr Anaesth 2014;24:1204–11.[doi:10.1111/pan.12490](http://dx.doi.org/10.1111/pan.12490)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1111/pan.12490&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=25039321&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 

16. Hoffman JR, Schriger DL, Mower W, et al. Low-risk criteria for cervical-spine radiography in blunt trauma: a prospective study. Ann Emerg Med 1992;21:1454–60.[doi:10.1016/S0196-0644(05)80059-9](http://dx.doi.org/10.1016/S0196-0644(05)80059-9)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1016/S0196-0644(05)80059-9&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=1443841&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=A1992JZ89300009&link_type=ISI) 

17. Stiell IG, Wells GA, Vandemheen KL, et al. The Canadian C-spine rule for radiography in alert and stable trauma patients. JAMA 2001;286:1841–8.[doi:10.1001/jama.286.15.1841](http://dx.doi.org/10.1001/jama.286.15.1841)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1001/jama.286.15.1841&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=11597285&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=000171595300024&link_type=ISI) 

18. Workman JK, Ames SG, Reeder RW, et al. Treatment of Pediatric Septic shock with the surviving Sepsis campaign guidelines and PICU Patient Outcomes. Pediatr Crit Care Med 2016;17:e451–e458.[doi:10.1097/PCC.0000000000000906](http://dx.doi.org/10.1097/PCC.0000000000000906)
    
    

19. Long E, Duke T. Fluid resuscitation therapy for paediatric sepsis. J Paediatr Child Health 2016;52:141–6.[doi:10.1111/jpc.13085](http://dx.doi.org/10.1111/jpc.13085)
    
    

20. Maitland K, Kiguli S, Opoka RO, et al. Mortality after fluid bolus in African children with severe infection. N Engl J Med 2011;364:2483–95.[doi:10.1056/NEJMoa1101549](http://dx.doi.org/10.1056/NEJMoa1101549)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1056/NEJMoa1101549&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=21615299&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 
    
    [Web of Science](http://emj.bmj.com/lookup/external-ref?access_num=000292199800004&link_type=ISI) 

21. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011;104:510–20.[doi:10.1258/jrsm.2011.110180](http://dx.doi.org/10.1258/jrsm.2011.110180)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1258/jrsm.2011.110180&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=22179294&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 

22. Tavender E, Haskell L, Oakley E, et al. Translating Research Evidence Into Paediatric Emergency Medicine: An International Network. 5th Annual NHMRC Symposium of Research translation. Melbourne. Australia: Australian Government, 2016.
    
    

23. Babl FE, Borland ML, Phillips N, et al. Accuracy of PECARN, CATCH, and CHALICE head injury decision rules in children: a prospective cohort study. Lancet 2017;389:2393–402.[doi:10.1016/S0140-6736(17)30555-X](http://dx.doi.org/10.1016/S0140-6736(17)30555-X)
    
    

24. Cunningham CT, Quan H, Hemmelgarn B, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol 2015;15:1–8.[doi:10.1186/s12874-015-0016-z](http://dx.doi.org/10.1186/s12874-015-0016-z)
    
    [CrossRef](http://emj.bmj.com/lookup/external-ref?access_num=10.1186/1471-2288-15-1&link_type=DOI) 
    
    [PubMed](http://emj.bmj.com/lookup/external-ref?access_num=25555466&link_type=MED&atom=%2Femermed%2Fearly%2F2017%2F08%2F30%2Femermed-2017-206727.atom) 

25. Smith J, Keating L, Flowerdew L, et al. An emergency Medicine Research Priority Setting Partnership to establish the top 10 research priorities in emergency medicine. Emerg Med J 2017;34:454–6.[doi:10.1136/emermed-2017-206702](http://dx.doi.org/10.1136/emermed-2017-206702)
    
    [Abstract/FREE Full Text](http://emj.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzNC83LzQ1NCI7czo0OiJhdG9tIjtzOjUwOiIvZW1lcm1lZC9lYXJseS8yMDE3LzA4LzMwL2VtZXJtZWQtMjAxNy0yMDY3MjcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)