You are here

Article Text

Article menu
Review
Safe staffing for nursing in emergency departments: evidence review
Free
  1. Alejandra Recio-Saucedo1,
  2. Catherine Pope2,
  3. Chiara Dall'Ora1,
  4. Peter Griffiths1,
  5. Jeremy Jones3,
  6. Robert Crouch4,
  7. Jonathan Drennan2
  1. 1National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (Wessex), University of Southampton, Southampton, UK
  2. 2Faculty of Health Sciences, University of Southampton, Centre for Innovation and Leadership in Health Sciences, Southampton, UK
  3. 3Faculty of Medicine, University of Southampton, Southampton Health Technology Assessment Centre, Southampton, UK
  4. 4Emergency Department, University Hospitals Southampton NHS Trust, Southampton, UK
  1. Correspondence to Dr Alejandra Recio-Saucedo, University of Southampton, Building 67, Highfield, Southampton, SO17 1BJ, A.Recio-Saucedo{at}soton.ac.uk

Abstract

Objective Getting staffing levels wrong in hospitals is linked to excess mortality and poor patient experiences but establishing the safe nurse staffing levels in the emergency department (ED) is challenging because patient demand is so variable. This paper reports a review conducted for the National Institute for Health and Care Excellence (NICE) which sought to identify the research evidence to inform UK nursing workforce planning.

Design We searched 10 electronic databases and relevant websites for English language studies published from 1994. Studies included reported a direct measure of nurse staffing relative to an activity measure (eg, attendances, patient throughput) or an estimate of nurse staffing requirements. Randomised or non-randomised trials, prospective or retrospective observational, cross-sectional or correlational studies, interrupted time-series, and controlled before and after studies were considered.

Results We identified 16 132 items via databases and 2193 items through manual and other searching. After title/abstract screening (by one reviewer, checked by a second) 55 studies underwent full assessment by the review team. 18 studies met the inclusion criteria for the NICE review, however 3 simulation studies that reported simulated rather than measured outcomes are not reported here.

Conclusions The evidence is weak but indicates that levels of nurse staffing in the ED are associated with patients leaving without being seen, ED care time and patient satisfaction. Lower staffing is associated with worse outcomes. There remain significant gaps and in particular a lack of evidence on the impact of staffing on direct patient outcomes and adequate economic analyses to inform decisions about nurse staffing. Given that an association between nurse staffing levels and patient outcomes on inpatient wards has been demonstrated, this gap in the evidence about nurse staffing in EDs needs to be addressed.

  • nursing, emergency departments
  • emergency department

http://dx.doi.org/10.1136/emermed-2015-204936

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Inadequate staffing levels in hospitals has been linked to excess mortality and poor patient experiences.1–4 Safe nurse staffing requires that there are sufficient nurses with the required skills to meet patient needs, and that they are organised and managed in a way that enables them to deliver the highest quality of care possible. Establishing a safe nurse staffing level is a key challenge, notably in the emergency department (ED) where the acuity and quantity of patient demand is highly variable. Demands on EDs nationally and internationally are increasing. In England, the number of patients attending consultant-led 24-h EDs with full resuscitation facilities has risen by approximately 12 per cent.5 Waiting times in major EDs have also risen: the percentage of patients seen in 4 h or less fell from 93.5% in 2013/2014 to 88.9% in 2014/2015. The number of patients waiting on a trolley for admission increased from 33 909 in the winter of 2010/2011 to 105 770 in the winter of 2014/2015.6 These increasing demands have implications for safe staffing nursing requirements.

    We conducted an evidence review for the National Institute for Health and Care Excellence (NICE) to identify research that could inform nursing staff requirements to support safe care in EDs across the UK. The review followed established NICE methodology for developing public health guidance7 and while directed towards UK policy, it includes international evidence to inform global debates about safe nurse staffing.

    For the purpose of this review, we defined EDs as consultant-led 24-h services with full resuscitation facilities and designated accommodation for Accident and Emergency patients (sometimes referred to in the UK as a Type 1 ED). The nursing team was defined as those delivering ‘hands on’ nursing care to adults and children, to meet fundamental needs and providing technical care, including medication administration and necessary administrative work. Nurse staffing focused on the size and skill mix (number of registered nurses in proportion to healthcare assistants) of the nursing team relative to the number of patients cared for, expressed as nursing hours per patient day (the number of hours worked by registered nurses and healthcare assistants divided by the number of patient hours over a 24-h period), nurse patient ratios or an equivalent measure.

    Our remit, agreed with NICE, was to review the literature reporting studies at department and organisation level to address six questions:

    1. What patient outcomes are associated with safe nurse staffing?

    2. What patient factors affect nursing staff requirements (eg, case mix and volume, acuity, dependency and other risk factors, including psychosocial complexity and safeguarding, informal (family) carer support, triage score and turnover)?

    3. What staffing factors affect nursing requirements including services provided by staff who are not part of the core nursing establishment, division of tasks, models of nursing care (eg, triage, rapid assessment and treatment), nursing experience, skill mix and specialism, transfer duties within the hospital and to external specialist units, nursing team management and administration, proportion of temporary nursing staff, and supervision and teaching?

    4. What environmental factors affect nursing staff requirements including availability and physical proximity of other units and specialties/ services, department size and physical layout and department type (eg, major trauma centre)?

    5. What organisational factors influence nursing staff requirements, including availability of other services or assessment models (ie, medical assessment units, ambulatory facilities or inpatient ward), crowding, management structures, organisational culture, policies and training?

    6. What approaches for identifying nursing staff requirements, including toolkits, are effective, reliable and/or valid and how frequently should they be used?

    For all questions we also considered relevant economic evaluations.

    Methods

    Search strategy

    We searched 10 electronic databases (Embase, CINAHL, CENTRAL, HTA, CDSR, DARE, NHS EED, NHS Evidence, Econlit and Medline) and seven relevant websites (American Nurses Association, Royal College of Nursing, Joanna Briggs Institute, Royal College of Emergency Medicine, Society for Acute Medicine, Faculty of Emergency Nursing, Trauma Audit & Research Network). Internet searches for grey literature and additional citation searching were also undertaken. A list of search terms is provided in the online supplement.

    Inclusion criteria

    The review covered literature published from 1994. To be included, studies had to report a direct measure of nurse staffing (eg, numbers of nurses on a shift, nursing hours per day) relative to a denominator based on activity (eg, attendances, patient throughput) as an independent variable or an estimate of nurse staffing requirements as a dependent variable. We considered randomised or non-randomised trials; prospective or retrospective observational studies; cross-sectional or correlational studies; interrupted time-series; controlled before and after studies. We included research published in English, and undertaken in the Organisation for Economic Co-operation and Development area (as per the NICE remit).

    Studies focused on service redesign or reconfiguration, and those which looked exclusively at other members of the multidisciplinary team, including emergency nurse practitioners or advanced nurse practitioners, were excluded. Likewise, we did not examine studies of single specialty EDs (ophthalmology or dental) or non-consultant (attending) led minor injuries units. Since aggregated findings allow the identification of issues at organisation level, potentially concealing variation at the level of units within organisations, work about nursing workforce planning or recruitment at network, regional or national levels was excluded.

    Quality appraisal

    A quality appraisal checklist based on the risk of bias assessment for cross-sectional studies published by NICE7 which was used in a previous review on safe staffing in acute care settings8 was used to assess risk of bias. Initial screening consisted of rapid exclusion based on title/abstract completed by one reviewer with a random 10% check by a second researcher. Any disagreements were resolved by recourse to a third independent reviewer. Studies were rated for internal and external validity separately and corroborated by two researchers.

    Search results were downloaded into the reference management software Endnote. Data were extracted on study aims, context/setting, research design, sample type and size, patient/nurse level risk adjustment, intervention, outcomes, conclusions. Summary tables of extracted data were produced and synthesised in a narrative form.

    Results

    We identified 16 132 items via databases and 2193 items through manual and other searching. After title/abstract screening (by one reviewer, checked by a second) 55 studies underwent full paper assessment and 18 studies met the criteria and were included in the final review (see figure 1). Three simulation studies that reported simulated rather than measured outcomes are not reported here as we did not have access to details of the primary data collected, analyses and estimated relationships used to develop these simulations.

    Figure 1
    Figure 1

    Study selection flow chart.

    Figure 2 illustrates the dispersed evidence currently available to inform nurse staffing in ED settings. Some studies looked at more than one outcome; most outcomes were examined in single studies, making it difficult to identify trends or meaningful patterns.

    Figure 2
    Figure 2

    Range of outcomes measured.

    Below we present the evidence provided by the studies structured around the research questions.

    Patient outcomes associated with nurse staffing (review question 1)

    Nine studies explored the relationship between outcomes and nurse staffing9–17 (table 1). The majority of these (six out of nine studies) were observational studies undertaken in single ED departments, which received between 30 000 and 180 000 patients per year. Seven out of nine studies were completed in the USA. Most of the studies were assessed as high risk of bias, limited on internal (five out of nine) or external validity (eight out of nine). A particular risk of bias associated with many studies was that the relationships reported are influenced by endogeneity (outcomes and staffing levels are influenced by patient need), which could result in attenuated staffing outcome associations or apparent counterintuitive results whereby higher staffing levels are associated with worse outcomes.

    View this table:
    Table 1

    Patient outcomes associated with nurse staffing*

    Outcomes reported included patient waiting times, time spent in the ED, patients who left without being seen, patient satisfaction, medication errors, time to aspirin or antibiotic administration, and ambulance diversion. The evidence regarding patient waiting times is mixed. However, there is evidence that lower levels of ED staffing are associated with increased levels of patients leaving without being seen. Studies in the USA11 found short-staffing of registered nurses to be a predictor of a higher number of patients leaving without being seen (OR 2.4; 95% CI 1.3 to 4.5, p≤0.006); the number of nursing vacancies (full time equivalent (FTE)) was strongly correlated with the percentage of patients leaving without being seen (r=0.89, p=0.007)12 and registered nurses to patient ratio was significantly associated with odds of patients leaving without being seen (OR 6; 95% CI 2.3 to 15.4).13

    Time spent in an ED bed before discharge or transfer to an inpatient bed was reported to increase10 when nurse staff was out-of-ratio (ratios defined as 1:1 for trauma resuscitation patients, 1:2 for critical patients, 1:4 for all other ED patients), with 37% longer care time (95% CI 34% to 41%, p<0.001) in two sites combined. Increases in nurse staff skill mix was associated with increased patient satisfaction.15 Longer lengths of stay for patients in ED were associated with an increase in hospital occupancy rates, additional patients admitted to the wards and the number patients admitted to intensive care unit (ICU) from the ED.16 No association was found between ED nurse staffing medication errors, time to antibiotics, ambulance diversion or patients’ time in the ED.

    Staffing, patient, organisational or environmental factors, and nurse staffing requirements (review questions 2–5)

    With regards to other staffing, patient, organisational, and environmental factors affecting nurse staff requirements, two studies18 ,19 (table 2) explored the introduction of a specialist psychiatric nursing service and absenteeism (staffing), one study20 explored workload and patient acuity (patient factors) and one21 explored the association between hospital-wide bed capacity, nursing and physician numbers at hospital level and waiting time (organisational factors). All four studies had significant risk of bias due to limitations in internal validity, and three out of four studies were assessed as having limitations in external validity. There were no studies that provided evidence regarding environmental factors.

    View this table:
    Table 2

    Staffing, patient, organisational or environmental factors and nurse staffing requirements

    A US study about task allocation20 analysed 63 nursing shifts and found that on average registered nurses spent 25.6% of their time performing direct patient care, 48.4% on indirect patient care, 6.8% on non-registered nurse (RN) care and 19.1% on personal time (meal and toilet breaks, reading, non-patient-related conversation). Personal task allocation varied with increasing direct and indirect patient care, with non-RN care remaining relatively constant, and decreasing personal time.

    One UK study assessed the impact of a dedicated specialist psychiatric nurse service on patient outcomes using a before and after cross-over design, assessed as weak for internal and external validity.18 This staffing intervention had no association with waiting times (hospital 1 p=0.76 and hospital 2 p=0.76), repeat attendances or patient satisfaction levels for people with mental health problems; however, there was evidence of difference of the referral of patients with mental health problems seen by the psychiatric nurse service when compared with the preintervention period. Patients seen by a psychiatric nurse who were recognised as having mental health problems were more likely to be transferred to a mental health unit than admitted to a medical ward (p<0.001), referred to an outpatient clinic (p=0.027) and less likely to be discharged against medical advice (p=0.001). The study found no association with waiting times, repeat attendances or satisfaction levels for mental health patients.

    A retrospective observational study,21 assessed as weak in terms of internal and external validity, modelled the impact of changing organisational variables on patient care time (time between being seen by a doctor and being admitted to hospital). This reported that a 1% change in the mean number of nurses at hospital level was associated with a 2.4% fall in ED waiting time and that an increase of 1% in the bed capacity was associated with a 3% fall in waiting time.

    We found no studies regarding the influence of environmental factors such as physical layout on nurse staffing requirements.

    Effective approaches for identifying nursing staff requirements (review question 6)

    Two studies reported on toolkits to determine staffing levels in the ED22 ,23 (table 3). One23 compared nursing work required against the actual number of nurses in the shift, but did not provide sufficient information to assess the reliability or validity of the tools used. The second tested the validity, reliability and generalisability of the Jones Dependency Tool (JDT)22 and identified a significant correlation between JDT and the nurses’ subjective ratings of patient dependency (R=0.786, p<0.001). There was a positive relationship between the amount of time spent by nurses in direct care of patients and the patient's level of dependency (R=0.72, p<0.001). The study also identified a relationship between JDT scores measured over time (Cohen's κ=0.68) as well as acceptable levels of inter-rater reliability between JDT and nurses’ subjective rating (κ=0.75). However, there was no external validation other than the subjective rating of staff adequacy and no measure of association with outcomes.

    View this table:
    Table 3

    Approaches for identifying nursing staff requirements

    Discussion

    This review identified a number of outcomes that appear to be associated with nurse staffing levels in ED. While the evidence is not strong, it appears to indicate that levels of nurse staffing in the ED are associated with patients leaving without being seen, ED care time and patient satisfaction. Lower staffing is associated with worse outcomes. We did not find strong evidence that waiting times, medication errors, and the rate of aspirin administration or ambulance diversion are affected by nurse staffing levels. There is conflicting evidence from two weak studies on the association between nurse staffing and time to antibiotics for patients with pneumonia. There was evidence from one study that patients with mental health problems seen by a dedicated psychiatric nurse in an ED were more likely to be transferred to a mental health unit.

    At an organisational level, longer lengths of stay for patients in ED were associated with an increase in hospital occupancy, additional patients admitted to the wards and the number patients admitted to ICU from the ED.16 One study,21 identified that increases in the number of nurses and doctors, and bed capacity in a hospital was associated with a reduction in the average waiting time in ED. Another study20 showed that as workload increased, direct and indirect patient care also increased while personal time decreased. There is no evidence of the effectiveness of toolkits for identifying staffing requirements although there is a suggestion that the JDT can be used to determine nursing workload in EDs.22

    This review has some limitations. The focus and scope of the review was determined by the remit provided by NICE and was necessarily tailored to the UK policy environment. Nonetheless our findings and conclusions have relevance beyond this context. We limited our search to studies in English, from Organisation for Economic Co-operation and Development countries, and only explored those that reported on the observed associations between staffing levels and patient outcomes. We have not reported on simulation and modelling studies as we did not have access to details of the primary data or relationships used to develop these. However the three simulation studies we examined for the larger NICE review do not alter the conclusions offered in this paper. All the studies reviewed were observational, no randomised controlled trials were identified and this is a significant weakness of study design and therefore of the evidence base.

    Furthermore, research exploring associations between staffing levels and outcomes needs to explore thresholds which might help identify adequate staffing levels, rather than only modelling linear associations. The patient populations and outcome measures varied across the research reviewed, as did the quality of the included studies which makes generalisation problematic. Synthesis of findings was not possible, in part due to the diverse outcomes studied (see figure 2). While clearly no single outcome can fully reflect safe and effective staffing in emergency departments, future research would benefit from a more consistent approach to measurement which may include utilisation (waiting times and left without being seen), safety (e.g. drug administration errors) and measures of staff wellbeing (job satisfaction and staff turnover).

    Conclusion

    There remain significant evidence gaps, notably a lack of evidence on the impact of staffing on direct patient outcomes such as mortality, failure to rescue, never events, time to pain assessment or falls. This is in stark contrast to the evidence base for the association between ward based nurse staffing and patient outcomes, which is large and offers strong evidence that lower nurse staffing levels are associated with higher rates of mortality and failure to rescue in North America, Europe and elsewhere24 ,25–28 although evidence that this relationship is causal and for relationships with other outcomes remains limited.29 Moreover there is no adequate economic evidence that could inform decision making about nurse staffing in EDs. Given compelling evidence of association between nurse staffing levels and patient outcomes on inpatient wards, further research is urgently needed to guide decision making about nurse staffing in EDs.

    Acknowledgments

    The authors thank Karen Welch, Information Scientist, for conducting the literature searches.

    References

    1. ↵
      1. Berwick D
      . A promise to learn—a commitment to act. Improving the safety of patients in England. National Advisory Group on the Safety of Patients in England. London: Stationery Office, 2013.
    2. ↵
      1. Francis R
      . Independent inquiry into care provided by mid Staffordshire NHS Foundation Trust January 2005-March 2009. Vol. 375. The Stationery Office, 2010.
    3. ↵
      1. Francis R
      . The Mid Staffordshire NHS Foundation Trust Public Inquiry—Chaired by Robert Francis QC. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. Executive summary. London: TSO, 2013.
    4. ↵
      1. Keogh B
      . Review into the quality of care and treatment provided by 14 hospital trusts in England: overview report. NHS, 2013.
    5. ↵
    6. ↵
      1. Blunt I,
      2. Edwards N,
      3. Merry L
      What's Behind the A&E ‘Crisis’? Nuffield Trust, 2015.
    7. ↵
      NICE. Methods for the development of NICE public health guidance—third edition (2012). National Institute for Health and Clinical Excellence, 2012.
    8. ↵
      1. Griffiths P,
      2. Ball J,
      3. Drennan J, et al
      . The association between patient safety outcomes and nurse/healthcare assistant skill mix and staffing levels & factors that may influence staffing requirements (NICE Evidence review). UK: NICE, 2014.
    9. ↵
      1. Weichenthal L,
      2. Hendey GW
      . The effect of mandatory nurse ratios on patient care in an emergency department. J Emerg Med 2011;40:76–81. doi:10.1016/j.jemermed.2009.02.037
      OpenUrlCrossRefPubMed
    10. ↵
      1. Chan TC,
      2. Killeen JP,
      3. Vilke GM, et al
      . Effect of mandated nurse-patient ratios on patient wait time and care time in the emergency department. Acad Emerg Med 2010;17:545–52. doi:10.1111/j.1553-2712.2010.00727.x
      OpenUrlCrossRefPubMed
    11. ↵
      1. Brown L,
      2. Arthur B,
      3. Lynch W, et al
      . Impact of nursing short-staffing and emergency department left without being seen. Ann Emerg Med 2012;Conference(var.pagings);60;4:S97.
      OpenUrl
    12. ↵
      1. Hoxhaj S,
      2. Moseley MG,
      3. Reese CL
      . Nurse staffing levels affect the number of emergency department patients who leave without treatment. Acad Emerg Med 2004;11:459. doi:10.1197/j.aem.2004.02.215
      OpenUrl
    13. ↵
      1. Greci LS,
      2. Parshalle CE,
      3. Calvitti A, et al
      . CrowdED: crowding metrics and data visualization in the emergency department. J Public Health Manag Pract 2011;17:E20–8. doi:10.1097/PHH.0b013e3181e8b0e9
      OpenUrlCrossRefPubMed
    14. ↵
      1. Chan TC,
      2. Vilke J,
      3. Killeen D, et al
      . Impact of mandated nurse-patient ratios on time to antibiotic administration in the emergency department. Ann Emerg Med, 2009;Conference(var.pagings):3.
    15. ↵
      1. Daniel I
      . The relationship between nurse staffing and patient satisfaction in emergency departments. Canada: University of Toronto , 2012:209.
    16. ↵
      1. Rathlev NK,
      2. Obendorfer D,
      3. White L, et al
      . Time series analysis of emergency department length of stay per 8-hour shift. West J Emerg Med 2012;13:163–8. doi:10.5811/westjem.2011.6743
      OpenUrlCrossRefPubMed
    17. ↵
      1. Schull MJ,
      2. Lazier K,
      3. Vermeulen M, et al
      . Emergency department contributors to ambulance diversion: a quantitative analysis. Ann Emerg Med 2003;41:467–76. doi:10.1067/mem.2003.23
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Sinclair L,
      2. Lazier K,
      3. Vermeulen M, et al
      . How effective are mental health nurses in A&E departments? Emerg Med J 2006;23:687–92. doi:10.1136/emj.2005.033175
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Green LV,
      2. Savin S,
      3. Savva N
      . “Nursevendor Problem”: personnel staffing in the presence of endogenous absenteeism. Manag Sci 2013;59:2237–56. doi:10.1287/mnsc.2013.1713
      OpenUrlCrossRef
    20. ↵
      1. Hobgood C,
      2. Villani J,
      3. Quattlebaum R
      . Impact of emergency department volume on registered nurse time at the bedside. Ann Emerg Med 2005;46:481–9. doi:10.1016/j.annemergmed.2005.07.014
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Harris A,
      2. Sharma A
      . Access block and overcrowding in emergency departments: an empirical analysis. Emerg Med J 2010;27:508–11. doi:10.1136/emj.2009.072546
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Crouch R,
      2. Williams S
      . Patient dependency in the emergency department (ED): Reliability and validity of the Jones Dependency Tool (JDT). Accid Emerg Nurs 2006;14:219–29. doi:10.1016/j.aaen.2006.06.005
      OpenUrlCrossRefPubMed
    23. ↵
      1. Korn R,
      2. Mansfield M
      . ED overcrowding: an assessment tool to monitor ED registered nurse workload that accounts for admitted patients residing in the emergency department. J Emerg Nurs 2008;34:441–6. doi:10.1016/j.jen.2007.06.025
      OpenUrlCrossRefPubMed
    24. ↵
      1. Kane RL,
      2. Shamliyan TA,
      3. Mueller C, et al
      . The association of registered nurse staffing levels and patient outcomes: systematic review and meta-analysis. Med Care 2007;45:1195–204. doi:10.1097/MLR.0b013e3181468ca3
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Rafferty AM,
      2. Clarke SP,
      3. Coles J, et al
      . Outcomes of variation in hospital nurse staffing in English hospitals: Cross-sectional analysis of survey data and discharge records. Int J Nurs Stud 2007;44:175–82. doi:10.1016/j.ijnurstu.2006.08.003
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Aiken LH,
      2. Sloane DM,
      3. Bruyneel L, et al
      . Nurse staffing and education and hospital mortality in nine European countries: a retrospective observational study. Lancet 2014;383:1824–30. doi:10.1016/S0140-6736(13)62631-8
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Duffield C,
      2. Diers D,
      3. O'Brien-Pallas L, et al
      . Nursing staffing, nursing workload, the work environment and patient outcomes. Appl Nurs Res 2011;24:244–55. doi:10.1016/j.apnr.2009.12.004
      OpenUrlCrossRefPubMed
    28. ↵
      1. Griffiths P,
      2. Jones S,
      3. Bottle A
      . Is “failure to rescue” derived from administrative data in England a nurse sensitive patient safety indicator for surgical care? Observational study. Int J Nurs Stud 2013;50:292–300. doi:10.1016/j.ijnurstu.2012.10.016
      OpenUrlCrossRefPubMed
    29. ↵
      1. Griffiths P, et al
      . The association between patient safety outcomes and nurse/healthcare assistant skill mix and staffing levels and factors that may influence staffing requirements (NICE evidence review). University of Southampton Centre for innovation and Leadership in Health Sciences, 2014.

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter Follow Alejandra Recio-Saucedo at @lexia14

    • Contributors CP, PG, RC, JJ, JD designed the review and developed the protocol. JD supervised the review. All team members were involved in independent screening, review and assessment of papers and data extraction. AR-S and CDO retrieved papers, coordinated the review activities, and synthesised the evidence. All authors were involved in drafting and editing the manuscript.

    • Funding The work reported here is drawn from a review initially conducted under a contract for NICE. Members of the research team are funded by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (CLAHRC) Wessex. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the CLAHRC, NIHR, NHS or the Department of Health.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement The full report of the NICE review is available at (https://www.nice.org.uk/guidance/indevelopment/gid-sgwave0762/documents). Full citations for all papers included in the review are provided in the manuscript.

    Linked Articles