As reported in the accompanying paper, a whole system review of the provision of emergency care was undertaken in Nottingham to inform local decision making about ways to improve system capacity and responsiveness.¹ As part of the review, we investigated the strength of patient preferences for particular generic characteristics or “attributes” of out of hours emergency care services. Concern for patients’ experience of health care is central to the government’s strategy for reforming emergency care, but giving consumers a greater say in the way the NHS works requires a concerted effort by decision makers to establish their preferences. In this study, local decision makers needed to be able to predict how patients might respond to local service reforms based on a valid preference measure for out of hours emergency care.

The out of hours environment has undergone considerable change in recent years. For example, growth in the number of NHS entry points (front doors) through which emergency care can be accessed and new roles for non-medical staff enabling them to contribute and alleviate pressures on the medical staff. These two components alone can be configured into quite different services. It is important that decision makers identify the best way to provide these services that should entail establishing how proposals for change might reflect patients’ preferences.

This paper describes the economic technique of discrete choice experiments that quantifies the intensity of preferences to ascertain information about relative satisfaction. The main objectives of the study were to provide information on the importance of several key attributes of out of hours services, their relative importance, how much patients were willing to trade between them, and the importance of preference heterogeneity.

BACKGROUND

The discrete choice technique was chosen because of its advantages over other methods; namely it provides information on the attributes of a service, on intensity of preferences, and asks the question in a realistic context; that is that it incorporates a notion of sacrifice.² This means that, unlike simple quantification of preferences, asking respondents to choose between options forces them to value attributes against each other. Application of this technique can help the decision maker to identify the important factors likely to influence preference.³

The technique presents respondents with alternative descriptions of a service, differentiated by different combinations of attribute levels (see below). Respondents are asked to choose their most preferred alternative. For each choice they make, the alternative selected is assumed to yield a higher level of satisfaction than that rejected. This enables the probability of an alternative being chosen to be modelled in terms of the attribute levels used to describe the service. There are two key underlying assumptions: the service can be separated into distinct attributes⁴ and satisfaction can be modelled as a behavioural response that comprises a systematic and random component.^5,6

The technique has been shown to be reliable and valid (for an overview of healthcare applications see Ryan and Gerard⁷). Two studies have applied the technique to the delivery of out of hours services in the UK.^8,9 The first found the single most important attribute was “whether the doctor seemed to listen”; to the extent that some respondents would not “trade” this attribute to obtain more of another.⁸ The authors concluded that improvements in doctor-patient communication might be most important when deciding how best to upgrade services. The second study also confirmed the importance of the attribute “doctor’s manner” in the care patients wanted to receive but here subjects were more prepared to trade this for more of other attributes such as reductions in waiting times and where the patient was seen.⁹ One unexpected finding was subjects’ relative dislike for telephone consultations. This is pertinent given the current trend to extend reliance on telephone led out of hours services. It should be noted that both these studies were undertaken before the introduction of important new initiatives (particularly NHS Direct and NHS walk-in centre). They are thus unlikely to represent the full range of choices relevant to today’s out of hours environment.

METHODS

The discrete choice technique comprises four steps; establishing the attributes and levels; selecting alternative scenarios to present (or which attribute level combinations); establishing preferences, and estimation.⁸

Attributes and levels

The process of defining attributes and levels was informed by published literature; national and local policy initiatives; guidance from the project’s steering committee; interviews with key stakeholders; and a pilot study. Attributes were selected that were generic to the varied components of the local emergency out of hours care system. Six attributes, each with three plausible and trade-able levels, were defined to characterise the initial contact for medical advice or treatment. Each is described below and presented in table 1.

The attribute “making contact” deals with how contact is started if the person needs medical advice that cannot wait until the next available GP appointment. This contact could be “in person” or “by telephone”. Recognising the government’s vision to see all out of hours partnerships operating integrated call systems by 2004¹⁰ we defined two levels of telephone contact; a single call whereby NHS Direct picks up the call (or integrated system) or multiple calls (caller is not automatically transferred to NHS Direct). As no hypothesis could be made for the preferred direction of the levels of preference, the categories were incorporated as two dummy variables using multiple calls as the comparator (1_CALL, INPERSON).

Medical advice can take place in the person’s own home or entail travelling to an NHS facility. We opted to use distance to measure location. The levels were “at home”, which entailed no travelling; or travel to the nearest NHS facility that, given local circumstances, could be, on average, 5 or 15 miles away. It was hypothesised that people would prefer less travelling.

Expected waiting time referred to time between the initial contact and receipt of medical advice or treatment. This was 0.5 hours, 2.5 hours, and 4.5 hours wait based on typical waiting times and desirable targets.¹¹ Previous studies support the hypothesis that shorter waiting is preferred.^12,13

Another attribute was concerned with whether patients were kept informed of the expected wait. This was predicated on the belief that patients would want this information either to allay anxiety or to allow their time to be put to better use, or both. This is endorsed by the government¹⁰ and supported by a previous study.¹²

Evidence is growing of the contribution that nurses and other professionals allied to medicine can contribute to the care of patients throughout the out of hours period.^14–16 Given that these roles may still be novel to patients, an important attribute under investigation was who advises the presenting patient. As it was difficult to postulate a priori whether other health professionals would be as preferred as the doctor, the categories “specialist nurse”, and “paramedic” were incorporated into two dummy variables that used paramedic as the comparator (NURSE, DOC).

Finally, there is evidence that the quality of the consultation experienced is important, such as being able to talk to the doctor and understanding the doctor’s explanations.^8,9 Ideas in the literature were adapted to signal a salient generic notion of the quality of the consultation received. This was presented as three ordinal levels using the notion of sufficient uninterrupted consultation time. It could be hypothesised that better quality would be preferred.

Selecting scenarios

There are a total of 729 possible attribute level combinations or scenarios. Not all of these could be used in a questionnaire so a software program was used to generate a fractional factorial sample.^17,18 These were paired and placed into choice sets.¹⁹ The properties associated with the choice sets are reported elsewhere.²⁰ The most appropriate decision context was to ask the respondent to imagine they needed urgent medical advice when the GP surgery was closed. Table 2 shows an example of a choice presented.

Establishing preferences

The questionnaire comprised two sections: section 1 presented 10 choices comparing out of hours “Service A” and “Service B”; and section 2 asked about sociodemographic details, experience of out of hours care and health services generally and current health. In addition responders were asked how easy or difficult they found the choice questions. The questionnaire was produced in English, but arrangements were in place for translation if required. Previous research recommends between 30 and 100 respondents per subgroup.^17,21

The study population comprised individuals who were attending for emergency treatment or advice with a key front door provider in Nottingham during February 2002. These were: A&E department; GP services (surgeries for same day appointments and two primary care emergency centres); NHS Direct and NHS walk-in centre. The NHS Direct survey was ongoing throughout the data collection fortnight, for other services time periods for distributing questionnaires were selected to cover likely busy periods when there would be patients waiting for treatment. Adults were considered to be individuals over 16 years of age, and adults accompanying younger patients were eligible to represent the views of younger respondents.

With the assistance of the staff in each location, patients who were waiting to be seen at A&E, GP services, or NHS walk-in centre were approached to take part in the study but only if they appeared not to be overly distressed, anxious, pre-occupied, too ill, or in too much pain. Patients with an injury to a hand or arm that would make it impractical to fill out the questionnaire were also excluded. Patients gave written consent to participate in the study, for which ethics approval had been obtained. A 60% response rate was anticipated.²²

NHS Direct callers were surveyed by post and a response rate of 40% was anticipated based on the average response from two previous studies.^9,23 The postal survey involved call handlers screening to check callers were from Nottingham and were not too ill, unduly anxious, or distressed. If none of these applied, callers were invited to participate in the research at the conclusion of their consultation. Verbal consent was given for NHS Direct to send out the survey and the completed questionnaire was returned directly to the research team. For ethical reasons it was not possible to follow up non-responders with reminder letters.

Analysis

The data were analysed using the probit model (although account was taken of repeated measurements of the data—respondents answered multiple choice questions—a random effects model was not significantly different). Only consistent responders were included. Consistency was assessed by analysing responses to particular choices (so called “dominance tests”).

A preference function was estimated based on the relative change in satisfaction associated with the differences in attribute levels for each choice. It took the form:

where V is the change in satisfaction associated with moving from Service A to Service B, 1–8 are the parameters of the model to be estimated, and e and u are the unobservable error terms where e is due to differences in observations and u is due to differences among respondents.

RESULTS

Of 620 questionnaires distributed, 457 (74%) were returned completed or partially completed and 378 (61%) were useable. The response rate varied, 33% was obtained from the postal survey and between 73% and 98% from remaining front doors (table 3).

Compared with the Nottingham population overall, this group of respondents had a higher proportion of women and 18–45 year olds but more similar proportions of ethnic groups and unemployed²⁴ (table 4). Most respondents had found the questionnaire easy or not difficult to complete.

Discrete choice model

Table 5 reports the results of the model for consistent responders. With some qualification, the size of the coefficients can be used to indicate relative importance per unit change of an attribute, and the signs on the coefficients indicate in which direction satisfaction changes. Care has to be exercised in interpreting these results however as they are clearly dependent on which units of change are adopted, and need to be considered alongside what is feasible by way of the scale of change. Allowing for this caveat all attributes influenced preferences for the group of responders as a whole. That is to say, and in order of importance, intensity of preferences were influenced by the extent to which:

• contact was with a doctor as compared with a paramedic;

• contact was with a nurse, as compared with a paramedic;

• being kept informed about expected waiting time with respondents preferring more information to less;

• quality of the consultation mattered with respondents preferring enough uninterrupted time to discuss their problem than less;

• making contact was in person was preferred, as compared with multiple telephone calls;

• making contact using an integrated call system was preferred to multiple telephone calls;

• location mattered with respondents preferring less travel (and by implication) consultation at home; and

• waiting time between initial contact and being advised or treated with respondents preferring less waiting time.

It is further helpful when trying to decide how to provide better services to consider the trade offs between attributes. In this study respondents were willing to forgo different amounts of waiting time to gain improvements in various attributes. For example, the typical respondent was prepared to wait an extra 2 hours 20 minutes to be consulted by a doctor (calculated by dividing the coefficient for DOC by the coefficient for WAIT_TIME).

Preference heterogeneity

Preference heterogeneity was investigated. It was found that compared with the results in table 5, younger respondents (those less than 45 years) had stronger preferences for making contact using an integrated call system and for being seen by the doctor. In contrast, older respondents appeared indifferent. All respondents were more likely to choose a system where they can be consulted by a doctor or a nurse rather than a paramedic, with only a slightly stronger preference for the doctor than nurse.

DISCUSSION

This study used a discrete choice experiment to measure strength of patient preferences for generic attributes of out of hours emergency care services. The results suggested all attributes were significant predictors of preferences and could be used to guide local modernisation plans. The most important attribute was being advised by a doctor with respondents’ expressing a willingness to wait over two hours longer to be advised by a doctor. The next two important attributes were being advised by a nurse and kept informed of expected waiting time. When thinking about making contact with a service, respondents preferred making contact in person to two or more calls and single call access to making multiple calls. A more general interpretation could be suggested from the data. Respondents would be prepared to tolerate not having services located closer to home if they could be advised by a doctor or nurse and kept better informed about expected waiting times. Such trade offs mean that service providers can consider more flexible packages of service locations on the proviso they also address issues around who provides advice and keeping patients informed.

The study had a number of limitations. The sample surveyed had a larger proportion of women and younger people than would be expected for the population of Nottingham, though not necessarily than would be expected among patients seeking urgent medical care.²⁵ However, under-representation of elderly healthcare users and those in poor health could be of greater concern for individual emergency services such as the A&E department and ambulance service, which may have sicker and more elderly populations. The study focused on users of emergency care services for whom the scenarios and choice options were particularly pertinent; however, a wider, more representative community survey may have produced different findings. Response to the postal survey was hampered because the researchers were not permitted to follow up respondents and remind them to return questionnaires. It was considered unethical to have access to callers’ names and addresses and the alternative, follow up by NHS Direct, was not feasible. This was an unfortunate limitation, as a higher response rate could have been anticipated if one or two reminders were used.^24,25 Instead the representativeness of NHS Direct callers must be called into question.

A further limitation concerns how respondents perceived the attributes that were described. In particular, how and where respondents anticipated their waiting time, which could not be ascertained from this study. This somewhat restricts the interpretation that can be placed on the importance of waiting time and being kept informed of waiting time. Waiting time spent at home next to the telephone is likely to be valued differently from waiting time spent in an A&E department. Likewise, being kept informed of waiting time is likely to be more valuable if you are free to use that time, which is not usually the case in A&E departments. A better understanding of how these attributes are perceived can be obtained with qualitative research methods. This should be a topic of future research.

This research builds on earlier studies to represent a more complete range of service choices that are relevant in the provision of today’s more complex out of hours health care environment. Similar to the findings of these previous studies qualitative aspects of the system were shown to be important to respondents.^8,9,22 In particular it was shown that people were concerned about being kept informed about expected waiting time and having contact with medical advice that was perceived as being long enough and free of interruptions. As such, some improvements to the system could be achieved by examining the way in which providers’ communicate with their patients. This could be straightforward to achieve, at relatively low cost.

The strongest preference elicited was being advised by a doctor rather than an alternative professional although the use of specialist nurses was also acceptable. One general interpretation is that initiatives that use or aim to use new (radical) ways of working, such as extending the role of paramedics, may yet be less acceptable in out of hours settings and need careful handling if they are introduced or expanded so that people, perhaps unfamiliar with them or suspicious of their effectiveness, are appropriately informed of their place in the system and of the anticipated benefits. This study further suggested that, if preference heterogeneity is taken into account in modernisation plans, services for older people may have to be tailored differently from those for younger people.

Although waiting time appeared less important than most of the other attributes, both the scale of the measurement unit and setting needs to be kept in mind when interpreting the findings. This is particularly so given that possible improvements in time savings can be expected to impact differentially across the system. For example, long waits in A&E could be substantially reduced through initiatives such as streaming patients^9,25,26 but only quite small improvements could arise from an already responsive NHS Direct service.

In summary, respondents have preferences for how out of hours health care is organised and when presented with relevant choices about this in a DCE context can validly express the strength of their views. This should encourage healthcare decision makers. It is possible to measure strengths of preferences even for services not yet operational, and this is particularly helpful in policy analysis of possible future service configurations.