Article Text
PDF
Abstract
Introduction In February 2014, all 23 National Institute for Health Research medical research specialities were failing to meet recruitment targets, with ‘Injuries and Emergencies’ research performing particularly poorly. In this paper, the multicentre AHEAD study was used to explore issues surrounding recruitment in UK emergency departments.
Method The AHEAD study investigated management and outcomes in over 3000 anticoagulated patients who suffered a head injury. Data from the study were used to compare patient recruitment at 33 Type-1 emergency departments. A questionnaire was sent to a research nurse at each of these sites and 30 replied (91% response rate). The survey investigated the difficulties encountered during patient recruitment and whether these were related to recruitment methods. More detailed interviews were conducted with three research nurses, to gain further insight into the barriers and facilitators involved.
Results Overall recruitment varied widely between sites with an eightfold variation in recruitment rates. Population demographics and other uncontrollable factors will partly contribute to this variation. However, research nurses reported many problems, including site resources, lack of staff engagement and flaws in recruitment strategies, which could be improved.
Conclusions Many of the barriers to recruiting patients for research studies encountered by research nurses have previously been reported in the literature, but there remain consistent problems. Until solutions are found, researchers will continue to miss recruitment targets and this will have implications for the efficiency and quality of emergency medicine research in the UK.
- research, methods
- emergency departments
Statistics from Altmetric.com
Key messages
What is already known on this subject?
All medical specialties are missing National Institute for Health Research patient recruitment targets and emergency medicine research performs particularly poorly. The 2011 Government Plan to Growth implemented changes to address these issues but following these changes little improvement in recruitment has been seen.
What might this study add?
Recruitment data from the recent multicentre AHEAD study, along with surveys and interviews of research nurses, have been used to identify the current issues affecting patient recruitment in emergency care research.
Introduction
Despite numerous attempts to improve the situation, medical research studies are failing to meet their patient recruitment targets.
When planning a medical research study, a target sample size is calculated. Enough patients must be recruited to produce meaningful, reliable and statistically significant results. But for many years, researchers have struggled to recruit sufficient patients to reach their targets. Indeed ‘slower than expected recruitment’ is one of the most common problems encountered during research trials.1
A 2006 study found that only 31% of the 114 UK trials assessed successfully met the recruitment target and only 55% achieved >80% of the recruitment target.2 In total, 31% of the trials reduced their recruitment target, while 54% asked to extend their time for recruitment.
The problem is not limited to the UK and has been occurring for many years.
A 1984 study reported that 34% of trials conducted in the USA recruited <75% of their planned sample size.3 ,4 More recently, a 2006 study reported that 85% of US trials do not conclude on time due to poor patient accrual and 30% of trials fail to recruit a single participant.5
These recruitment failures affect the quality of research. If insufficient patients are recruited to a study, the researchers may decide either to analyse the reduced number of patients or to extend their trial in the hope of attracting additional volunteers. The former approach lessens the statistical power of a study, reducing the value of the results and their impact on clinical practice. However, if the time or number of sites for recruitment is extended, the cost of the study will increase and funding may be insufficient.2 Poor recruitment may result in cancellation of an entire study.
Emergency medicine is one specialty that is performing particularly poorly. In 2012, just 28% of UK emergency medicine studies recruited to the highest level, considerably lower than the National Institute of Health Research target of 80%.6
This study explores the barriers encountered during recruitment to the AHEAD study, a large multicentre study conducted in English and Scottish emergency departments (EDs), and aims to draw on this experience to make recommendations for future multicentre emergency medicine studies.
Methods
The AHEAD study, led by the School of Health and Related Research, University of Sheffield, involved research into the management and clinical outcomes of patients taking the anticoagulant warfarin who suffered a head injury. The study involved 33 Type-1 EDs and aimed to recruit 3000 patients, a goal that was achieved. Clinical data on all potentially eligible patients were collected at the time of presentation. A follow-up questionnaire was distributed to all surviving patients 6 weeks later, at which time they were informed of the study.
During the design of the study, a waiver of consent, approved in accordance with section 251 of the National Information Governance Board for Health and Social Care was granted for the initial collection of patient-related data. The waiver facilitated recruitment and meant that follow-up could be conducted after patients had left the ED. Patients could formally withdraw from involvement in the study.
This paper analyses recruitment data from the AHEAD study, allowing a comparison of recruitment methods and trends between the various EDs during the 18-month period. Each site planned and organised its own recruitment strategy and the methods adopted were not mandated by the study protocol, so this paper analyses the effectiveness of these processes. Site data have been anonymised.
A survey was emailed to research nurses following the completion of recruitment. The key sections of the survey related to the following:
The organisation of patient data within the ED.
Whether clinical data were stored manually or electronically.
The method for recruitment to the AHEAD study.
Any difficulties encountered, and any changes that occurred in the ED that influenced recruitment strategy.
To gain additional insight into patient recruitment, three research nurses agreed to be interviewed. The main themes of the interviews were as follows:
What are the barriers to recruitment in emergency care research?
How are recruitment strategies planned for patient recruitment?
Why is it harder to recruit in emergency medicine than other specialties?
What could be done to improve patient recruitment?
The objective of the interviews was to identify and evaluate issues not highlighted in the surveys.
This research was covered by the ethical approval granted to the AHEAD study (reference 11/H1308/13).
Results
It was originally estimated that 10 Type-1 EDs would recruit 3000 patients over a period of 15 months (September 2011–December 2012). However, an audit undertaken in two hospitals prior to the commencement of recruitment found that the number of patients that fitted the criteria for the AHEAD study was much lower than expected and that 10 EDs would not reach the target in the time allocated. After discussion with the project management group, it was decided that the study should instead be conducted at 33 sites. Despite increasing the number of sites, it was still necessary to extend recruitment by 3 months to March 2013, making the total recruitment period 18 months long.
This enabled the AHEAD study to successfully reach its recruitment target with 3720 eligible patients identified. Of these, 154 patients subsequently requested they be withdrawn from the study and 3566 patients were enrolled.
The number of patients recruited at each site varied widely, but differences in the dates of study commencement meant that the recruitment rate was a better indicator of how effectively a site was able to enrol patients. The recruitment rate was calculated as the total number of patients recruited/total number of recruitment days. The mean recruitment rate for the whole study was 0.26 patients/day, but the inter-departmental range was 0.08–0.66. Figure 1 demonstrates no association between site launch date and site recruitment rate. Through interviews and surveys, other possible reasons for the variation between sites have been identified.
Recruitment rate at each site by month of site launch.
Thirty sites (91%) returned the survey. The majority of questions on all surveys were answered, with ‘On average, how long does it take to identify an eligible patient?’ the only question frequently omitted; it was unanswered on 9 of the 30 returned surveys (30%). Furthermore, there was wide variation in the responses to this question, suggesting that it was interpreted in different ways.
Recruitment rate will have been affected by the size of the ED and the demographic of the population it serves. Due to their greater risk of falling, head injuries occur frequently in the elderly, and such patients are more likely to be anticoagulated. The median age of patients recruited to the AHEAD study was 81 years, so departments situated in areas with a high elderly population will probably see more eligible patients. Demographic factors are beyond the control of research staff but will have a significant impact on recruitment.
Differences in recruitment strategy
Survey responses demonstrated variations in the organisation of EDs. Twenty different hospital information systems were in use at the 30 sites surveyed. There was also wide variation in the manual and electronic methods for storing and accessing patient notes and clinical data. Consequently, each site had to use a different method to identify and recruit eligible patients. These variations had a big impact on the time required to identify eligible patients: one site reported that 1 week's worth of patients could be identified in 30 min; another stated that identification and recruitment required 1–2 h daily.
Most sites used a combination of manual and electronic methods to identify patients, but the effectiveness of electronic searches differed greatly. While site 8 acquired a new search programme during the study that produced a list of potential patients within seconds, the system at site 15 required 1–2 h to be dedicated to recruitment daily.
Site 11 trialled the use of an iPad ‘app’ to improve staff involvement in the study. This method was found to be successful and significantly improved recruitment by clinical staff.7
Problems with eligibility
Researchers at several sites encountered difficulties during recruitment as a result of diagnostic summaries. A common issue was that patients with multiple trauma including head injury did not always have head injury listed as a presenting condition. Consequently, those sites which searched for patients on the basis of presenting symptom found that eligible patients were being missed. Once this issue was identified, researchers were required to search through past patient notes again to ensure that all eligible patients were included.
Impact of staff on recruitment
A common problem in emergency medicine research recruitment is the reliance on non-research staff to identify and consent patients. In the AHEAD study, research staff were less dependent on clinical staff as patients could be identified retrospectively and there was no face to face consent process required. However, some sites did encounter difficulties when involving other staff in the recruitment process. Site 30 had no ED attendance log and so researchers were reliant on staff to inform them of eligible patients. The site-30 survey states that despite employing methods such as posters and departmental meetings to raise awareness, clinical staff were responsible for only 10% of the total number of patients recruited.
There is rapid staff turnover in EDs and each new group of staff must be educated about the process of a research study. Additionally, in the AHEAD study over 25% of the dedicated research staff changed. New staff may be unfamiliar with the research and recruitment process and this will lead to inefficiency in recruitment.
Research site alterations
System changes that influence the efficiency of a recruitment strategy can occur. A new electronic system at site 19 meant ED data could be accessed anywhere in the hospital, which was viewed as a very positive change by the research nurse. In contrast, at site 33 the system changed such that paper copies of patient notes were scanned into a computer system and destroyed after 1 week. While a search of paper records took 30 min, identifying all patients using the computer system took up to 2 h—a particular problem if the research staff had been absent for a period of leave.
Site 3 started scanning all patient notes onto an electronic database, which saved time for their research nurse. However, when the same system was implemented at site 14 the research nurse struggled to find the relevant electronic notes.
Informed consent
The waiver of consent for AHEAD clinical data meant that issues commonly encountered with obtaining informed consent in an ED were not encountered during this study and is likely to have contributed significantly to its success.
The research nurses commented that obtaining consent for research studies is often difficult in an ED. At site 27, the research nurse stated that ‘the informed consent process often has to take place in busy waiting rooms, the lack of dedicated space for research adds to the difficulty of consenting scared and emotional patients’. The site-26 researcher believed that ‘one of the main problems when recruiting is the patients perception of research, which is often negative, possibly due to media influence’.
Overall the questionnaires and interviews showed that various factors, both uncontrollable and avoidable, can influence the recruitment of patients across sites.
Discussion
Recruiting patients to take part in emergency medicine research is difficult. Investigations over the past 15 years have shown that the barriers include the following: study design, staff co-operation, patient acceptance and the mechanisms for obtaining consent.8–13 The recruitment of patients to the AHEAD study demonstrates the extent to which these issues occur in a recent UK multicentre emergency medicine study.
Study design
Research studies fail if the investigators overestimate the numbers of patients eligible for inclusion, or underestimate the time and difficulties of recruiting them. Overly stringent inclusion or exclusion criteria exacerbate the problems.
The AHEAD study successfully recruited its required number of patients, but the study took longer and involved more sites than expected. It was many months after the start of the trial before some sites recruited their first patient (figure 1).
Multicentre studies must take into account potential variations in recruitment rate between different sites that may be caused by uncontrollable factors such as the size and demographics of the population served by the site.
The process used to recruit patients to a trial must be efficient and effective. A small pilot study permits some of these issues to be identified and addressed before large amounts of money are invested in a major study, but this is not always practical, and results from one site may not be generalisable. In the AHEAD study, it was not possible to provide comprehensive advice to units, as every ED had different practices and systems. Each site had to develop its own recruitment strategy, but sites identified as recruiting well were asked to share their processes with other sites.
Any flaws in study design must be addressed promptly, as it may be difficult to resolve issues as the study progresses. Only 53% of UK trials conduct a pilot study, inadequate preparation may be one of the reasons why so many studies are missing recruitment targets.2
Staff barriers
Emergencies occur at all times of day and night and dedicated research staff cannot be present all the time. Clinical and administrative staff are therefore essential to the success of a project but must be persuaded of its value and be able to dedicate the necessary time to the task. In the AHEAD study, patients were identified retrospectively and this may have contributed to its success, as recruitment could usually be done entirely by research staff. However, some sites had poor information systems and still relied on clinicians and administrative staff to select the patients. If, for reasons of time or enthusiasm, staff didn't want to become involved in the research, recruitment was poor.
Strategies to increase the interest and involvement of clinical staff in research studies need to be investigated. Efficient information systems help considerably by reducing the administrative burden. Including mention of the research within clinical protocols and briefing new staff effectively may reduce the disruption caused by staff turnover.
Informed consent barriers and patient barriers
The Declaration of Helsinki requires all research participants to give informed consent unless the study meets the strict criteria that allow a waiver of consent.
The requirement for informed consent creates huge problems for research in emergency medicine. As a result of illness, trauma and anxiety, the proportion of patients in an ED capable of giving fully informed consent may be as low as 20%.14 ,15 Furthermore, few of the patients are accompanied by someone who could give consent on their behalf.9 Issues with consent greatly reduce the efficiency of the recruitment process.
Even if a patient is capable of giving consent they may choose not to do so. Patients’ perception of the risks and benefits of a study have a major influence on recruitment to a study.11 When interviewed, a research nurse said that one study had almost 100% of eligible patients refuse due to the study protocol.
The AHEAD study researchers were granted a waiver of consent for all clinical data. Patients were informed of the study once they had left the department, at which point they could choose to formally withdraw from the study; however, only 154 people choose to do this—a very low rate compared with most studies. Furthermore, the ability to collect patient data without consent meant that patients could be identified and recruited after they had left the department, thereby reducing the time that researchers had to spend in the ED and making them less reliant on clinical staff.
The design of the AHEAD study also meant that very little was required of participants and this is likely to have contributed to the low withdrawal rate.
Site barriers
Analysis of the AHEAD study uncovered issues not previously identified in the literature associated with the individual sites at which recruitment was taking place.
The research nurse surveys demonstrated that the facilities available in an ED such as electronic patient databases have considerable effect on the time required to identify patients. Some sites lacked the facilities to rapidly identify appropriate patients. Good information systems can improve efficiency, but poorly implemented systems can create havoc.16 At some sites, changes to systems improved the recruitment, while at other sites the recruitment process had to be redesigned and took far longer than previously.
Involving researchers when change is implemented at a research site could improve efficiency and reduce the problems that occur if research staff are forced to change their recruitment strategies.
The 2011 UK Government Plan for Growth stated that research staff and site funding would be dependent on their ability to meet recruitment goals. The AHEAD study recruitment data show that a wide range of factors can slow recruitment at a site. Some of these factors are unavoidable while others are significant issues that are difficult to deal with once a study has started. Careful planning is essential when setting a recruitment target to ensure that the sample size is achievable.
The AHEAD study succeeded where other studies have failed, but even this study underestimated the difficulty of patient recruitment to large multicentre studies in emergency medicine. Key to its success was a protocol that minimised the involvement of busy clinicians whose priority is dealing with huge clinical workloads. Retrospective recruitment of patients and an opt-out consent strategy undoubtedly helped and this strategy may be applicable more widely.
Acknowledgments
The authors would like to thank all the participating sites contributing to the AHEAD study and the research staff who gave their time to be interviewed.
References
Footnotes
Contributors MK designed and distributed research nurse surveys and was responsible for correspondence with research nurses. MK also provided AHEAD study recruitment data. RJ was responsible for analysis of the data. SM assisted MK and RJ in study design and the overall co-ordination of the research. All authors listed made significant contributions to the final report and drafting of the article.
Funding This paper presents independent research funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0808-17148. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests None.
Ethics approval NRES Committee Yorkshire and The Humber—Sheffield.
Provenance and peer review Not commissioned; externally peer reviewed.