Knowledge translation during the COVID-19 pandemic

Evidence-based medicine (EBM) has been previously defined as ‘the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients’.1 EBM involves three pillars: published evidence, clinical judgement and the patients’ values and preferences. The COVID-19 pandemic has arguably been one of the greatest challenges to EBM since the term was coined in the last century. In our modern and highly connected world, this life-threatening disease has spread around the globe affecting millions of people in a matter of months. It is perhaps not surprising that the speed and severity of the illness has challenged the traditional models of knowledge translation, both from a public health perspective and with regard to direct clinical care.

Traditional, or ‘idealistic’ knowledge translation, models rely on a careful progression of investigation from pathophysiological theory, through animal models, then small human trials, subsequent larger randomised controlled trials (RCTs), repetition and eventual systematic review/meta-analysis of all available data, in order to provide a robust answer as to whether an intervention works, does not work or may cause harm.2 That process typically takes many years.3 In a pandemic, which by its very nature develops rapidly, clinicians may feel that we do not have time to follow this traditional EBM approach or that we should place more faith in expert advice or clinical judgement. However, we argue that this is a time to strengthen our understanding of EBM and to continue to apply its principles, although with greater vigilance and agility than normal.

Such challenges are not unfamiliar to emergency medicine clinicians. The evidence base for much of our practice is relatively weak, leading us to practise under the principles and practice of ‘best available evidence’; using the published literature to influence our practice, while highlighting its strengths and weaknesses through critical appraisal and pragmatic application. These principles underpin the long running BestBets programme4 published online and in the EMJ, which takes a pragmatic approach to clinical decisions recognising that at times the evidence to support our practice is limited.

As individuals, clinicians will have different thresholds at which they decide to change practice. This threshold is determined by many factors which extend beyond the simplistic notion of the strength of evidence. Our decisions will always be influenced by our personal values and attitudes to risk, the health economies we work in, where we sit in the decision-making hierarchy, our perception of potential benefits and risks and of course by the characteristics and wishes of the individual patient in front of us. The key point is that our decision to change is not simply a matter of objectively appraising the evidence. Our own thoughts, beliefs and experiences combine with potential clinical impact to affect our judgement. How vulnerable we as clinicians feel may also impact our decisions, especially when the impact of that decision might be perceived to negatively affect our self-image. For example, there are limited proven treatments for COVID-19, yet we know that many of our patients will die of it. Prescribing no medications may lead to a perception of therapeutic helplessness, which could then drive us to not just stand there, but ‘do something’5 almost certainly at a lower level of evidence than we might ordinarily use to exact a change in our prescribing practice.6 Even outside a pandemic situation, we have a tendency to overestimate the potential benefit on an intervention and underestimate the potential harm; this issue is undoubtedly compounded by the pace and scale of a pandemic.7

The nature of a pandemic is to push clinicians into a position where we must make decisions that are both ‘time critical and information light’. This dilemma will again be familiar to many emergency and critical care clinicians, who make similar judgements with individual patients in the resus room. However, as clinicians we are not used to making such decisions that may affect large groups of patients or health economies. At the time of writing, we are faced with critically ill patients on a daily basis who ‘might’ benefit from therapies that have theoretical, in-vitro or observational data on efficacy. What then should we do? How can we balance our hope in novel treatments versus our concern about potential harm? What should we believe and when should we change?

While this paper focuses on therapeutic interventions, our concerns, thoughts and solutions are equally applicable to diagnostics, prognosis and prevention articles.

Precipitous decisions in COVID-19

The COVID-19 pandemic has given us many examples of the traditional values of EBM being challenged by the urgency of the clinical situation. A range of treatments were initially proposed, all of which have a reasonable theoretical basis, but none of which had proven treatment effect in clinical trials. Despite this, reports from China, Italy and the USA describe large numbers of patients being prescribed early ‘compassionate use’ medication for which there is biological plausibility, but little or no evidence of effectiveness in humans. The vast majority of these prescriptions have taken place outside of clinical trials and thus we cannot know whether these interventions have made a difference.

Politicians too, perhaps in pursuit of good news at a time of worldwide anxiety, have advocated for many of these therapies to be made widely available, with the implication that there is little or no risk. Most notably, Donald Trump advocated that hydroxychloroquine (HCQ) should be widely prescribed.7 ,8 To date, we have still seen no clear evidence of clinical effectiveness.9 Similar hopes have also been placed on remdesivir, an antiviral drug that was not shown to have a statistically significant effect in an RCT published in the Lancet, but which 24 hours later was given emergency use authorisation by the US Food and Drug Administration.10 This approval was seemingly based on a secondary outcome of symptom duration (but not mortality), in an unpublished (at the time of the decision) RCT. Interestingly, the outcome measures of this trial were changed in the weeks before publication which may have also biassed the impact of the conclusions. In normal times, the unverified report from the RCT of remdesivir in the USA would barely cause a ripple of interest, yet during a pandemic it has precipitated worldwide media attention, presidential statements and regulatory approval.

These precipitous decisions have also extended into the scientific literature. The expedited thirst for information and the rapidity of the pandemic have led to abbreviated peer review, publication of unvalidated data, retraction and dissemination through press release.11–14 Indeed, one of the most important RCTs of the last decade announced a positive effect for dexamethasone through initial press statement, with immediate endorsement from the health secretary prior to any release of preprint information or opportunity for critical appraisal.14 15 Clinicians are left with challenging decisions on when and how to prescribe this therapy, basing their decision either on face validity and acceptance of authority-based medicine, or on clinical judgement and evaluation of online preprint data, still currently awaiting formal peer review.16

What is the harm in trying?

Change often depends on the type of intervention proposed. In times of crisis, relatively cheap and apparently safe interventions often form the first wave of attack. If something has face validity, no clear evidence of harm and is readily available without significant investment, it is perhaps obvious why clinicians will lower their change threshold to adopt it. However, this approach is often an appeal to emotion through flawed reasoning, and not without peril. No matter how cheap and plausible, interventions that do not work carry opportunity costs, offer false hope, distract clinicians from the pursuit of other beneficial therapies and introduce new potential for workplace error and harm.

A recent example comes in the form of vitamin C. Following a before and after single-centre service evaluation, high-dose intravenous vitamin C was lauded by many as a widely available, safe and low cost option for the treatment of that particularly controversial, heterogeneous and challenging disease, sepsis.17 The clinical community was divided, but several authors proposed a widespread adoption of this therapy based on limited efficacy data and it being unlikely to cause any harm. Fast-forward several years and recent trial data suggest no conclusive benefit from vitamin C,18 and no sign that the initial high reported success rates can be replicated in a trial setting.19 Have we done any harm by using it in the meantime? Possibly not directly, as the intervention itself carries relatively few side effects and risk. However, even if clinicians did not cause direct physical harm to patients by using this medication, use of such unproven therapies is likely to have wasted a lot of time and effort, in addition to the search for an answer delaying other viable research options. What were we not doing, while we were busy giving vitamin C to everyone?

Similar ‘low risk’ therapeutic interventions have emerged for COVID-19 and received international endorsement, despite an absence of evidence. Awake prone positioning is an example reported in small case series without control group and purported to transiently improve oxygenation.20 National organisations have immediately endorsed this strategy21 and other key advocates are busy designing physical support devices to facilitate the intervention. But does it work?22 Is it well tolerated by patients?23 24 Is it better than usual care (sitting someone out in a chair) and does any transient benefit in oxygenation actually persist? Early reports are observational, inconsistent in methodology and based on small uncontrolled case series data.20 25 Some authors report the need to maintain a vigilant level of monitoring during awake prone position which may be difficult to provide during a pandemic. Such potential harm may be difficult to detect in small and/or poorly designed observational studies.

Many will argue that there is no harm in trying interventions that at face value have few obvious risks. But the potential harms of distraction, false hope, suboptimal use of resources, misunderstanding of patient trajectory and potential clinical risk will always result from broad application of a therapy that has not yet been thoroughly investigated. There is also potential harm through an apparent loss of equipoise, which may hamper efforts to obtain the evidence required to determine whether the intervention is indeed effective in the future (eg, in future randomised trial proposals). We must remember these issues when we are deciding what to believe and when to change.

What would you want, if you had COVID-19?

This is a question often put forward by enthusiasts for a novel, untested therapy. If you were sick, would not you want the best available treatment options? This assumes that such treatments are always beneficial, not likely to harm and advocated by your treating clinician based purely on expertise and scientific rationale. Unfortunately, this is often not the case.

As we outlined above with regard to HCQ, this is a classic example of a treatment option with face validity which has been widely publicised by political bodies as an exciting, effective therapy. The actual scientific evidence has been less optimistic and we now believe that it is ineffective at best and possibly even harmful.9

Perhaps less classic has been the mission creep of widely available and licensed therapies. Concerns on clotting problems in COVID-19 has led to several papers reporting high prevalence rates of thrombosis among critically ill patients. Despite broad composite outcomes, verification bias, uncontrolled methodology and a limited comparison population, such studies have led to rapid changes in practice. Many centres have increased their dosing of pharmacological thromboprophylaxis based on unvalidated risk characteristics (such as arbitrary D-dimer cut-points) and others have opted for immediate therapeutic dose anticoagulation in those patients with severe illness.26 We have no evidence that either of these strategies are clinically effective. However, we do know that they have the potential to cause harm. The bleeding risks associated with therapeutic dose anticoagulation have been clearly established over decades of research. As such, we should be asking for a high level of evidence to support practice change, given we expect some resulting harm.27

Recommendations to adopt such therapies are clearly made with the best of intentions. Yet, it is prudent to ask if the science behind them is of an acceptable standard. National repository sources for information attempt to collate and distill the most relevant information for ease of presentation, but are again challenged by the urgent need for information and the developing literature base.28–30 Many national repositories have required regular updates throughout the pandemic and international guidance documents are often conflicting in recommendations.31 32 Such is the risk when combining limited evidence with expert opinion.

To answer our own question about what we would wish to receive should we fall unwell with COVID-19, then at the time of writing, our answer would be to not receive any therapies that are not evidence based. However, we would all be very happy to be enrolled in any clinical trial looking to evaluate such treatments, with appropriate governance and oversight as even if enrolled to the placebo arm outcomes are likely to be better than with non-trial treatment.33

The way forward

While we argue that the traditional view of EBM has been undermined during the pandemic, we also recognise the difficulties that we all face in a rapidly changing situation. The question is what can we do now to promote EBM, and what might we put in place for future pandemics, or during a resurgence of COVID-19. We suggest four strategies below.

1. There are many trials in progress at this time. We should do everything that we can to ensure that every patient with COVID-19 has the opportunity to enter a clinical trial. In the UK, all hospitalised patients should have the opportunity to enrol in a range of clinical studies such as ISARIC, PRIEST and RECOVERY, which have very broad inclusion criteria. At present, however, only a minority of eligible patients are being recruited to the RECOVERY trial which is assessing therapeutic interventions for COVID-19. Our aim during this time-critical pandemic should be for all patients to be given the opportunity to participate in clinical trials. All departments should work with research and innovation teams to ensure this happens. A list of current UK studies supported by the Chief Medical Officer and coordinated by the National Institute for Healthcare Research (NIHR) is available online.34

2. Ensure that research delivered during a pandemic is of the highest possible quality. A global pandemic is a time to raise the bar of science, not to lower it.35 Retrospective cohort and single-centre studies are highly unlikely to give us definitive and timely answers to important clinical questions. Effort and focus should be concentrated on prospective, multicentre, well-designed studies looking to directly assess causation/effectiveness and targeting patient-oriented outcomes.

3. Encourage the use of routinely collected, anonymised data to support epidemiological studies. In healthcare economies such as the NHS, it is now possible to collect anonymised electronic data (including both physiological and outcome data) from care records and electronic notes.36 Such data could be used to support machine learning/artificial intelligence techniques which may be able to answer or direct clinical questions earlier than traditional study methodologies.

4. Design studies for deployment in future pandemics and place them in a ‘hibernated state’ such that the research infrastructure is in place prior to requirement. This approach was taken in the UK by NIHR34 who supported the design of a number of trials for the next influenza pandemic. When COVID-19 emerged, several of these trials were repurposed, thus allowing the UK to launch nationwide clinical trials within a few weeks of cases arriving.

EBM in a pandemic

The urgency and severity of the COVID-19 pandemic contains threats and opportunities to clinicians wishing to practise EBM. It is arguably a time when we can experience first-hand the journey from ignorance about the disease through to a better understanding and approach to diagnostics and interventional therapy. In addition, national frameworks have facilitated a comprehensive and cohesive research effort in order to address a spectrum of issues focused around a single disease. This is groundbreaking; from public health studies on face masks, to interventional studies on complex immunomodulatory therapies, the clinical and academic communities have acted together to provide broad research opportunities. These efforts highlight both the importance of embedded research culture and the need for expert clinical judgement alongside.

However, one other key aspect in this digital age is the sheer volume of new ‘evidence’ that we are faced with. On the one hand, this research can be both informative and hypothesis generating, but on the other hand, it is prone to selective promotion and can overwhelm the user by the nature of volume and frequency of publication. For most clinicians alive today, the only comparable event was the emergence of HIV. Even this did not match the urgency that COVID-19 places on us.

The principles of EBM are more important now than at any other time in our careers. We must collectively do all that we can to ensure that our response to the pandemic is based on the science and not on the emotional, political or economic issues that challenge it. We echo the call of others for critical reasoning, critical appraisal and critical thinking during these challenging times.37