Article Text

PDF

An introduction to economic evaluation
  1. S Goodacre,
  2. C McCabe
  1. University of Sheffield, Sheffield, UK
  1. Correspondence to:
 Dr S W Goodacre, 87A Sydney Road, Sheffield, S6 3GG, UK;
 s.goodacre{at}sheffield.ac.uk

Abstract

An outline of the principles behind economic evaluation of emergency care.

  • economic evaluation

Statistics from Altmetric.com

Clinicians working in accident and emergency (A&E) medicine will have little difficulty accepting the idea that health service resources are scarce. Increasing demands for health care and limited resources with which to meet them are a familiar part of the emergency environment. All clinicians will be aware of the need to make difficult choices in deciding which health care interventions to fund.

Health economics tackles this problem of scarcity of resources and the implicit requirement to make choices that will maximise the benefit accrued from their consumption.1 It therefore entails far more than simply accounting or attempting to cut costs. Yet many of the concepts behind economic evaluation will be unfamiliar to practising clinicians. The aim of this article is to explain some of the basic ideas behind economic evaluation.

WHY DO WE NEED ECONOMIC EVALUATION?

Clinical trials measure health care outcomes to determine the efficacy or effectiveness of health care interventions. If resources are unlimited, this is the only information we require to decide which interventions to use. We simply choose the most effective option. However, because resources are limited we also need to know whether the intervention represents good value for money. In other words, is it cost effective?

WHAT IS AN ECONOMIC EVALUATION?

Simply measuring the costs of an intervention will not tell us whether it is cost effective. A cheap intervention may represent poor value for money if it has little effect on outcome. Economic evaluation is the process of measuring cost effectiveness.

An economic evaluation will measure two parameters—cost and outcome (effect). Because two parameters are measured, the results of an economic evaluation will not necessarily tell you which treatment option is “better” in the same way that a clinical trial might. If the cheapest option is also the most effective, it will clearly be the most cost effective. In this situation the most cost effective option is described as being dominant. However, if the cheapest option is not the most effective the decision of which intervention to choose is less clear. In this situation the results will typically take the form of an incremental cost effectiveness ratio, expressed as the additional cost incurred per additional unit of effect accrued.

When no intervention is dominant economic evaluation will tell you how much extra you will need to be prepared to pay to achieve an improved outcome. As such, health economics will inform decision making, rather than dictating a decision. The idea that economic evaluation is only about determining which is the cheapest option is a simplistic and mistaken idea. It is also a dangerous one as it risks losing the valuable insights that economics can provide.

OPPORTUNITY COST

The concept of opportunity cost is fundamental to health economics.1, 2 It is based upon the idea that scarcity of resources means that expending resources on one health care activity inevitably means sacrificing activity somewhere else. The opportunity cost of undertaking an activity is defined as the benefits that must be foregone by not allocating resources to the next best activity.

For example, you decide to employ a chest pain nurse in your department to achieve National Service Framework targets for thrombolysis. To do this you must make tough choices elsewhere. Perhaps you must do without some clinical assistant sessions? But this will cause overall waiting times to increase. The opportunity cost of employing a chest pain nurse is the benefit you must forego by being unable to fund the next highest option on your list of priorities. If your next highest priority is more clinical assistant sessions, then the cost of foregoing these (an increase in waiting times) will be the opportunity cost of employing a chest pain nurse.

THE ECONOMIC VIEWPOINT

Definition and measurement of the opportunity cost of an intervention will depend upon the economic viewpoint taken. In the above example, we have taken an extremely limited viewpoint—that of the A&E department. We are only concerned with the A&E budget and what it can fund. If we took a broader perspective, that of the whole hospital, or even the whole health service, for example, our opportunity cost might be different. The highest unfunded priority for the hospital might be a waiting list initiative to reduce the coronary artery bypass graft waiting list. Foregoing this initiative would be the opportunity cost of employing a chest pain nurse if the hospital viewpoint was taken.

The economic viewpoint could be the department, the hospital, the health service, or society as a whole. The choice of viewpoint taken for an economic evaluation should be determined by the question you wish to answer and should be made clear in any description of an economic evaluation.

If an economic evaluation were to be undertaken to inform the allocation of a fixed departmental budget, the departmental perspective would be appropriate. However, if the evaluation is to inform a case for extra funding for the department from within the hospital budget then the hospital perspective is the most appropriate. It is good practice to take as broad a perspective as possible (that is, societal). The societal perspective encompasses the data for all other perspectives and ensures that decisions are made with the best information possible. An overly narrow perspective can fail to identify important “knock on” effects that can impact upon the success or failure of an intervention in the long term. For example, an economic evaluation of NHS Direct that took the primary care perspective and ignored A&E might represent a poor information base for decision making about the future of NHS Direct.

ECONOMIC EFFICIENCY

We often talk about trying to make the most efficient use of our available resources. Achieving economic efficiency entails obtaining maximum benefit from our given resources. There are two types of economic efficiency—technical efficiency and allocative efficiency.2

Technical efficiency simply entails achieving a given objective with the least possible expenditure. If our objective is to reduce A&E waiting times, we could achieve this by employing more medical staff or by training nurse practitioners to assess and treat patients with specific complaints. The most technically efficient option will be that which reduces waiting times at the lowest cost. We are simply concerned with how we meet our specified objective—reducing waiting times.

Allocative efficiency entails deciding what objectives we will attempt to meet and the extent to which we will try to meet them. Determining allocative efficiency entails making a value judgement about the relative merits of different objectives. The example given earlier, where we had to decide whether to allocate resources to employing a chest pain nurse to achieve door to needle time targets for thrombolysis, or employ clinical assistants to reduce overall waiting times, is an example of a decision relating to allocative efficiency. It is not simply a matter of deciding which intervention will best meet our objective, we must decide which objective is most worthwhile meeting.

Questions of technical efficiency are unsurprisingly rather easier to answer than those relating to allocative efficiency. Economic evaluation may be used to inform and illuminate issues of allocative efficiency, but because a value judgement is required, decision making will ultimately be up to clinicians, patients, politicians, and health care managers. Health economic data may tell us how much we will need to pay to achieve our objectives—reducing door to needle times and reducing overall waiting times. Health economic data can also tell us what health benefits we might expect from achieving certain targets, such as how many lives will be saved by reducing door to needle time by a specific amount. More controversially, comparisons between interventions, such as those presented in a “marginal cost per QALY league table” (see below), can address some of the issues of allocative efficiency. Ultimately, however, deciding which benefits are worthwhile will entail some sort of value judgement.

WHAT ARE THE KEY FEATURES OF AN ECONOMIC EVALUATION?

An economic evaluation is a comparison of the costs and outcomes of health care interventions. As such it provides a measurement of economic efficiency. To be an economic evaluation a study must have two essential features:

  1. Both costs and outcomes must be analysed, and

  2. More than one alternative strategy must be compared.

It should be clear by now that economic evaluation is not simply a matter of measuring costs of interventions and then choosing the cheapest option. This is clearly foolish and not an approach any of us would take in our lives outside health care. If you wanted to buy a product you wouldn't simply buy the cheapest available regardless of quality, your personal preference, or whether it did what it was supposed to do. Equally, we do not always buy the best product available as we wish to have money left to buy food. Cost is important, but no more important than the outcome from the expenditure. It may ultimately be appropriate to choose on the basis of cost alone, but only if we can show that outcomes are equivalent.

Measuring outcome from health care interventions is one of the great challenges of health economics. It is astonishing, considering the amount of money we spend on health care, that our ability to measure benefit from health care is only recently receiving serious attention, and remains relatively crude. We collect reams of data showing how many new patients attend our department, how many investigations we do, and how many treatments we do. Yet is any of this of any benefit to our patients?

Just as we would not accept evidence of effectiveness without comparison to a control group, we cannot measure cost effectiveness without some sort of comparison. The choice of comparator may be difficult, because we want to choose the best alternative from the point of view of both costs and benefits. For this reason, if there is no alternative strategy that is of proven effectiveness, the most appropriate comparator may well be a “do nothing” alternative. Doing nothing certainly should not cost much, and, if there is no evidence that the intervention being investigated is effective, will be relatively cost effective.

TYPES OF ECONOMIC EVALUATION

Several types of economic evaluation are recognised.3, 4 It is the measurement of outcome that determines what type of economic evaluation has been performed.

COST MINIMISATION ANALYSIS

If the outcomes of the alternative strategies are demonstrated to be equivalent, then analysis will consist of simply comparing costs and choosing the cheapest option.5 Demonstration of equivalence of outcome may entail presentation of primary data from the study itself, or presentation of secondary data, such as the results of a meta-analysis.

While this sounds simple, cost minimisation analyses often conveniently ignore the issue of uncertainty surrounding the estimates of comparative effectiveness.6 Outcomes are extremely unlikely to be identical. More probably there is no statistically significant difference in outcome. All good clinical trials, and meta-analyses, should present their results with confidence intervals. These will probably include the possibility of the more costly option also being more effective. Estimating confidence intervals for cost effectiveness data is fiendishly difficult and well beyond the scope of this article.7 However, it is always worth examining the confidence intervals for outcome data presented with a cost minimisation analysis to see whether the more expensive strategy could still produce a worthwhile improvement in outcome.

COST EFFECTIVENESS ANALYSIS

For a cost effectiveness analysis, the outcomes of the alternative strategies are not equivalent and are measured in uni-directional natural units, such as lives saved, change in pain score, or change in peak flow rate.8 The results are therefore helpful in determining technical efficiency. They tell us which strategy maximises a given objective, such as improving pain score, with the lowest cost.

An example of a cost effectiveness analysis might be a randomised controlled trial comparing the costs and effects of two thrombolytic agents for acute myocardial infarction. A suitable primary outcome might be mortality at 30 days. If the cheaper thrombolytic agent were also the most effective, interpretation of the results would be easy and our choice of thrombolytic therapy clear. The cheaper, more effective agent would be dominant.

But what do we do if the more expensive agent is also the most effective? If we make our decision on the basis of cost we will choose the cheapest agent, whereas if we make our decision on the basis of effectiveness, we will choose the more expensive agent. The study can help our decision making by presenting results as a cost effectiveness ratio—in this case, the cost per life saved at 30 days. This tells us how much extra we must pay for each additional life saved and allows us to consider whether alternative uses of the same resources would generate more health benefits.

So how much should we be prepared to pay to save a life at 30 days with our more expensive thrombolytic agent? £10 000? £100 000? How about £1 million? This is, of course, a value judgement and depends upon many factors. Again it is worth emphasising that economic evaluation can inform and illuminate the decision making process, but cannot make the decision for us.9

Deciding whether to fund this expensive thrombolytic agent, it would clearly be useful to be able to compare our cost effectiveness ratio to estimates of cost effectiveness for other competing uses for our resources. Yet many interventions in A&E (or health care in general) will not affect mortality. How do we compare our cost per life saved to the cost per change in pain score of an expensive analgesic agent, or the cost per change in peak flow rate of a treatment for asthma? This is an important limitation of cost effectiveness analysis.

COST UTILITY ANALYSIS

For a cost utility analysis the outcomes of health care interventions are measured in units of health outcome that combine quality and quantity of life, and can thus be compared between different interventions and health problems.10 The most well known example of a measure of health utility is the quality adjusted life year, or QALY.11

Calculation of QALYs entails first measuring quality of life on a scale from zero to one, where zero equates to death and one equates to perfect health. The period of time (in years) over which this quality assessment applies is then multiplied by its quality weighting to give the number of quality adjusted life years.

Cost utility analysis therefore offers the attractive prospect of allowing comparison of a wide and varied range of health care interventions. This has lead to the development of “marginal cost per QALY league tables”, which compare the marginal cost per QALY of interventions as diverse as cholesterol screening and heart transplantation.11

There are many theoretical, methodological, and ethical concerns with these analyses that are beyond the scope of this article.12 Well conducted cost utility analyses comparing interventions within the same area of health care can be a powerful way of assisting decision making, but the use of marginal cost per QALY league tables to compare diverse health care interventions is highly controversial.

COST-BENEFIT ANALYSIS

Although the expression “cost-benefit” analysis is commonly used for any form of economic analysis, it has a very specific meaning in health economics. It refers to economic analyses in which the outcomes are valued in monetary units.13 As costs and benefits are both measured in the same units they can be compared directly. This is clearly very useful for assisting decision making. However, measuring benefits in monetary units presents substantial difficulties. While cost-benefit analyses have many theoretical attractions for health economists, they are unlikely to be frequently encountered in the medical literature.

COST CONSEQUENCES ANALYSIS

This is a form of cost effectiveness analysis.3 Ideally a cost effectiveness analysis will have a primary outcome that can be used to produce a cost effectiveness ratio. However, often more than one outcome is relevant and it is difficult to determine which is the most important. For example, a trial comparing the cost effectiveness of treatment for sprained ankle might consider ability to weight bear, pain score or several other outcomes to be equally important.

Combining several outcomes to create a single index of health utility is one option, but this may be insensitive to important differences in outcome. Another option is to present a cost consequences analysis. All important outcomes are presented with relevant cost effectiveness ratios and the reader is left to judge the relative importance of the outcomes. The limitation of this type of analysis is that it does not allow transparent assessment of whether the health gained from the expenditure of limited resources is being maximised. A decision maker can only be confident of this when one intervention dominates the others on all outcomes and cost.

THE MARGIN OR THE AVERAGE?

You may have noticed that, when considering cost utility analyses, we referred to marginal cost per QALY league tables, rather than average cost per QALY. The distinction between the average and the margin is important in all forms of economic evaluation.

The average cost per unit of output is the total cost divided by the total output. The marginal cost per unit of output is the cost of the next unit of output. In economic evaluation we are nearly always interested in the margin rather than the average. In fact, when describing cost effectiveness it should be apparent that the “average cost effectiveness” of an intervention is a fairly meaningless concept, because much of our outcome will be achieved without any intervention.

Taking our example of thrombolytic agents, most patients will survive to 30 days without any thrombolysis. We are only interested in those additional lives saved when a thrombolytic agent is compared with an alternative. Thus we are interested in the marginal cost per life saved—the additional cost required for our treatment to save a life that would otherwise have been lost.

From this, it can be seen that the choice of comparison therapy is an important factor in economic analysis. An experimental treatment can appear cost effective if compared with an inefficient alternative (either in terms of cost or outcome). Conversely, the development of an efficient way of delivering an alternative treatment can undermine the cost effectiveness of an intervention.

SUMMARY

In this article we have outlined some of the basic principles behind economic evaluation. The methodology of economic evaluation is developing; many issues need to be resolved and many controversies still exist. What is now beyond doubt is that, outside of the utopia of unlimited health care resources, economic evaluation will play a vital part in determining optimal management for our patients. Only by understanding the principles of economic evaluation can we achieve this in the A&E department.

REFERENCES

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.