• research, methods

Government funding for clinical research in the National Health Service (NHS) is channelled through the various parts of the National Institute for Health Research (NIHR).1 This is the one area of the NHS which is still receiving a real terms annual increase in funding despite the economic crisis. In exchange for this abnormal largess, the government is keen to ensure that the best possible value is derived from their money. The charity funders of medical research are also very keen to ensure that the donations that they receive are used as efficiently as possible, and industry wants to keep commercial research costs down. All of the sources of research funding therefore have a common objective: increasing efficiency in the delivery of clinical research.

The current main measure of efficiency is whether or not the study delivers to ‘time and target’; in other words, whether or not the researchers recruit the sample number of patients in the time that they said they would. Other performance ‘metrics’ are whether or not the study starts on the planned date (called ‘time to initiation’), and whether or not the researcher is entering data each month into the NHS Portfolio2 (called ‘upload engagement’: a process that has to happen every month, even if the number of patients recruited is zero). In order to calculate the targets for each research project the sample size, start date and planned recruitment time are taken from the information that the researcher submitted in the Integrated Research Application System (IRAS)3 form, so the performance targets are in fact set by the researcher themselves: being held to account against a target that you set yourself seems to be a very fair system.

The performance metrics are updated every month, with each study on the NHS Research Portfolio being ‘red/amber/green’ rated. The system is relatively new, but for the first time this year the data are now giving a way of tracking local research performance for every project in the NHS Research Portfolio. At present, Emergency Medicine does not do very well and is in the lower half of performance. We are included in the Injuries and Emergencies section which has only 28% of its studies rated ‘green,’ compared with 68% for the best performing group (Public Health). There is an overall target for the NHS to achieve 80% of its research recruiting according to the planned schedule (green on time and target). It is likely that future funding for clinical research from the government will depend on performance against these performance metrics.

As collection of these performance measures continues to improve they will be other potential uses. In future, if you are making a grant application the research funder is likely to use these metrics to assess your track record of delivery in previous projects. If you want to be a participating site in a multi-centre study, the Chief Investigator (or commercial sponsor) will want to know your delivery metrics for previous projects before picking you to be part of the new study. It is also likely that when considering a strategic investment in Emergency Medicine a research-funding organisation will want to know whether or not this is an area that can give efficient research delivery. This information has not been available in the past, but it certainly will be in the future. Given this background, it is important at both a personal and specialty level that our research projects perform well against these targets.

In order to help deliver Portfolio research the NIHR has set up the Research Networks (emergency medicine research falls within the Comprehensive Local Research Networks (CLRNs))4 and an independent peer support system, the Injuries and Emergencies National Specialist Group (I&E Group).5 The I&E Group looks at all emergency medicine research each month and contacts the Chief Investigator to see if any assistance can be given if the project is falling behind time and target, has not started recruitment by the Portfolio start date or is not uploading monthly recruitment data to the Portfolio. The most common form of support is to put the investigator in touch with the appropriate part of the local CLRN.

The I&E Group/CLRN system can help to overcome barriers to delivery, but it cannot help if the researcher has set themselves an unrealistic target in the first place. Emergency Medicine researchers seem to be consistently overoptimistic when predicting their ability to recruit patients. There may be practical reasons for this (such as moulding the application to fit a specific amount of funding available) or psychological reasons (such as overestimating the prevalence of a condition when inclusion and exclusion criteria are taken into account). Many Emergency Medicine clinical studies are undertaken by relatively inexperienced researchers (there is a very encouraging year on year increase in our Portfolio research activity), but this means that the full difficulties of patient recruitment may not be appreciated. Feasibility studies are time consuming and expensive, and so are seldom undertaken. Whatever the underlying reason we are too often setting ourselves a target that cannot be achieved.

Emergency Medicine researchers are also overoptimistic about the time that will be needed to undergo local governance procedures (the big remaining unreformed barrier to NHS research). This is of course not the researchers fault, but the appearance of poor delivery can be avoided if the time taken for local permissions is accurately estimated on the IRAS form. The dates initially supplied on the IRAS form are not set in stone—with the agreement of the funder of the research both the start and finish date can be altered by the researcher. This gives a system that is sufficiently flexible to take into account the various delays that a clinical research project can encounter. Unfortunately, researchers often do not make these changes to the information about their study on the Portfolio, and so potentially are penalising themselves for the future by appearing to have poor performance against the metrics.

The big current change within this system in 2012 has been to extend these metrics to a local level, so that rather than just looking at the overall recruitment and talking to the Chief Investigator it will be possible for the local I&E Group/CLRN (and others) to see how the study is performing locally and talk to, and support, the local Principle Investigator. This local level of detail about past performance will be useful to future researchers (both commercial and public funded) when they are choosing which centres to invite to participate in a new study (it is a pretty safe bet that in research past performance is a good guide to future performance).

At present, there is little awareness of the importance of these issues among Emergency Medicine researchers, probably because there is currently no penalty within the system for having a study that is slow to recruit. However, there are changes ahead which will mean that current performance against the time and target, study start date and upload engagement targets will have a significant effect on how much research takes place within our specialty and which centres will be involved. The ways in which this will take place are not yet explicit; however, we currently have an opportunity to be ahead of the game. As well as continuing to work on developing research infrastructure in Emergency Departments (with support from the CLRNs) and continuing to tackle the barriers to delivery of research (led by the College Research Committee and the NIHR I&E Group), the main change that is needed is for Emergency Medicine researchers to consider very carefully when filling in the IRAS form (both the main form and the Site Specific Information) the start date, sample size and finish date. These key fields may not have been seen as very important in the past, but they certainly will be in the future. Research performance metrics are here to stay and will affect the future of research within our specialty, so all research active emergency physicians need to be aware of their growing importance.