Article Text
PDF
Abstract
Introduction Emergency doctors routinely face uncertainty—they work with limited patient information, under tight time constraints and receive minimal post-discharge feedback. While higher uncertainty tolerance (UT) among staff is linked with reduced resource use and improved well-being in various specialties, its impact in emergency settings is underexplored. We aimed to develop a UT measure and assess associations with doctor-related factors (eg, experience), patient outcomes (eg, reattendance) and resource use (eg, episode costs).
Methods From May 2021 to February 2022, emergency doctors (specialty trainee 3 and above) from five Yorkshire (UK) departments completed an online questionnaire. This included a novel UT measure—an adapted Physicians’ Reaction to Uncertainty scale collaboratively modified within our team according to Hillen et al’s (2017) UT model. The questionnaire also included well-being-related measures (eg, Brief Resilience Scale) and assessed factors like doctors’ seniority. Patient encounters involving prespecified ‘uncertainty-inducing’ problems (eg, headache) were analysed. Multilevel regression explored associations between doctor-level factors, resource use and patient outcomes.
Results 39 doctors were matched with 384 patients. The UT measure demonstrated high reliability (Cronbach’s α=0.92) and higher UT was significantly associated with better psychological well-being including greater resilience (Pearson’s r=0.56; 95% CI=0.30 to 0.74) and lower burnout (eg, Cohen’s d=−2.98; −4.62 to −1.33; mean UT difference for ‘no’ vs ‘moderate/high’ burnout). UT was not significantly associated with resource use (eg, episode costs: β=−0.07; −0.32 to 0.18) or patient outcomes including 30-day readmission (eg, OR=0.82; 0.28 to 2.35).
Conclusions We developed a reliable UT measure for emergency medicine. While higher UT was linked to doctor well-being, its impact on resource use and patient outcomes remains unclear. Further measure validation and additional research including intervention trials are necessary to confirm these findings and explore the implications of UT in emergency practice.
- cost efficiency
- risk management
- psychology
- safety
- staff support
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Emergency medicine demands rapid decision-making with limited information where doctors’ tolerance for uncertainty may affect resource utilisation and clinical decisions.
WHAT THIS STUDY ADDS
We introduce a measure of uncertainty tolerance (UT) showing significant associations with improved doctor well-being but inconclusive results for resource use and patient outcomes.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Enhancing UT could improve doctor well-being suggesting potential benefits from integrating tolerance training in emergency departments. Further research is needed to replicate findings related to resource use and patient outcomes.
Background
Emergency doctors routinely face uncertainty; they work with limited patient information under tight time constraints and receive minimal postdischarge feedback. Indeed, to do their jobs they need to tolerate this uncertainty. Uncertainty tolerance (UT) means adaptively balancing thoughts, feelings and actions in uncertain situations.1 2 Research shows that people with higher versus lower UT tend to feel less anxious, think more clearly and make decisions less influenced by uncertainty.1 3–5 While some see UT as a context-dependent ‘state’, we believe it’s likely that while UT fluctuates, an underlying ‘trait’-like stability shapes how individuals handle uncertainty.6
Across various specialities, higher UT is robustly associated with better psychological and work-life related well-being among clinicians (eg, reduced stress; increased resilience).2 Supporting doctors to manage uncertainty may thus help reduce endemic burnout among healthcare staff7 8—particularly in specialties with ‘high rates of undifferentiated illness and uncertainty’ as a large recent survey emphasised.9 While mixed, a plurality of evidence also shows that clinicians with higher UT make decisions that lead to lower resource use (eg, admit fewer patients; order fewer tests).2 This, combined with the possibility that higher burnout in low UT doctors could result in poor staff retention and/or more safety incidents, has led some to hypothesise that supporting staff management uncertainty could improve efficiency.6 7
Nevertheless, UT remains underexplored in the unique emergency medicine context. Most recent work has explored how emergency doctors navigate uncertainty rather than examining associations between UT and factors like clinical decision-making.6 10–12 Past studies looking at UT associations13–17 have had limitations. First, they were USA-based—poorly reflecting emergency department (ED) dynamics of resource-constrained universal healthcare systems.18 Second, they focused on specific tests/treatments (eg, CT scans) and/or patient groups—limiting generalisability. Third, they used the Physicians’ Reaction to Uncertainty Scale (PRUS) to assess UT which is not emergency medicine specific and primarily assesses emotional reactions to uncertainty (eg, anxiety) missing others (eg, behavioural) and undermining validity.1 Lastly, they did not assess UT associations with indicators of safety (eg, postdischarge mortality) despite the possibility that higher UT may cause patient harm through risky decision-making (eg, inappropriate discharges).19
We developed a more comprehensive, modified PRUS for the emergency medicine context and conducted an initial evaluation to assess associations between UT, doctor factors (eg, experience; burnout), resource use (eg, episode costs) and patient outcomes (eg, readmission). Based on literature from other specialties, we predicted that higher UT would be associated with lower resource use and better well-being indicators (eg, lower burnout). The association between UT and patient outcomes was considered either way; higher tolerance could lead to more risky decisions or alternatively, higher tolerance may protect doctors from cognitive ‘overload’—providing better judgement of costs/benefits of interventions which could have a beneficial impact on resource use and patient flow.20
Method
Study design
We developed an UT measure and tested it among emergency doctors, assessing its association with their demographics (eg, years of experience, gender), a chart review of their recent patient management decisions and outcomes (eg, episode costs, reattendance rates) and personality/work life factors (eg, self-perceived burnout, confidence, resilience). Online supplemental file 1 provides a study design diagram.
Supplemental material
Measures
UT measure
To measure doctors’ UT, the PRUS21 22 was adapted to better suit the emergency medicine context. Hillen et al1 conducted a comprehensive analysis of UT in healthcare identifying 22 existing self-report measures including the most prominent: PRUS. They categorised items across these measures according to a taxonomy of uncertainty sources (eg, complexity, unpredictability, disorder) and three types of responses: cognitive (eg, denial), emotional (eg, stress) and behavioural (eg, avoidance).
The research team held discussions over a series of meetings to create a comprehensive and context-specific measure. Starting with the PRUS, we incorporated relevant items from other validated measures that addressed the sources and responses to uncertainty described in Hillen’s model that were not fully covered by the original PRUS. Items were also modified/excluded if they were not relevant to emergency medicine.
The measure was piloted with seven emergency doctors whose feedback led to further revisions enhancing clarity and ensuring the content was appropriate to the National Health Service. The measure asks respondents to rate agreement from strongly disagree (1) to strongly agree (5) on 34 items, including: ‘I usually feel anxious when I am not sure of a diagnosis’ and ‘The hardest thing to say to patients or their families is: I don’t know’. A score per doctor was derived by taking the mean (score range=1–5). It takes 7–9 min to complete. The measure can be seen in online supplemental file 2.
Supplemental material
Personality and work-life measures
We captured other factors including resilience (6 items), risk aversion (1 item), burnout (1 item), confidence (5 items), work life well-being (10 items) and experience of an adverse event at work (1 item). For further details and specific items see online supplemental file 3.
Supplemental material
Demographics
Doctors self-reported basic demographics including age, gender, hours worked a week, years of experience and seniority.
Patient selection criteria
The research team conducted an internal survey of 10 emergency doctors (from all sites except Leeds)—including members of the author team—ranking patient problems by greatest levels of uncertainty for the physician. These were used to guide the selection of patient charts to be reviewed for the association of a physician’s UT and patient management.
Doctor questionnaire
The final questionnaire was constructed through discussion and debate among the research team with some additional piloting work conducted with seven emergency doctors. In response to extensive pilot feedback, we kept the battery of measures brief using short and/or novel scales to reduce participant burden (eg, 1-item risk aversion scale). This pragmatic decision prioritised minimising slow recruitment, attrition and missing data over the potential loss of validity or reliability from using such measures.
The resulting questionnaire (online supplemental file 2) consisted of the UT measure, items about basic demographics (eg, age, grade), and the aforementioned personality and work-life measures. In total, it was 65 items and took 10–15 min to complete.
Participants
Doctors from five EDs in Yorkshire, UK completed all measures using an online questionnaire. Doctors at specialty trainee (ST) (year) 3 or equivalent and above were recruited only as they operate more independently than more junior doctors and thus make more independent and potentially impactful patient management decisions. Doctors were approached by collaborating colleagues and completed consent online (before the questionnaire). Recruitment commenced 18 May 2021 to 16 February 2022.
Patient episode outcomes
To correlate doctor responses on the questionnaire with patient-level variables, we manually extracted data from the 10 most recent patient encounters with each doctor from health records using a standardised form (online supplemental file 4). Eligible episodes met three criteria:
Supplemental material
The episode occurred 30 days (minimum) before doctors completed the questionnaire to avoid any influence of the study on their decisions.
The patient’s primary problem was abdominal pain, back pain, chest pain, headache or vomiting. These were the top five problems ranked as leading to uncertainty by the internal survey.
The patient was 18+ years old.
In addition to patient-level demographic and clinical variables, we extracted resource use and patient outcomes to assess their correlation with UT. Online supplemental file 5 provides details on all data items and our cost estimation procedures.
Supplemental material
Resource use indicators
Ordering of any test and/or treatment per patient episode.
24-hour or shorter stay length among admitted patients (‘unnecessary admissions’).
Estimated patient episode costs.
Patient episode costs were estimated using resource use and patient outcome indicators. The aim was not to estimate costs exactly but rather generate measures to assess variance in costs between doctors.
Patient outcomes
30-day attendance after initial attendance (‘reattendance’).
30-day admission to hospital on reattendance.
Though we captured mortality, it was not assessed as an outcome given just six deaths).
Participants
Sample size
Multilevel regression power primarily depends on the sample size at the higher level (level 2) which in our study refers to the number of doctors versus patients (level 1). This was a challenge because we aimed to assess doctor-level variables as predictors of patient-level outcomes and doctor recruitment was much more resource-demanding than episode data extraction. According to simulation estimates derived by inputting a range of plausible values into a web app,23 a sample of 500 patients across 50 doctors (achievable within our time constraints) provided sufficient power (70–95%) to detect small to medium effect sizes (Cohen’s d of 0.30 or 0.50) when comparing median split UT differences on binary outcomes.
Analysis
UT score distributions were assessed for variability and skewness. Cronbach’s α/McDonald’s ω assessed internal consistency and Pearson’s correlations, t-tests and analysis of variances (with Bonferroni-adjusted t-tests) were used to examine associations between UT scores and doctor-related factors.
Patient-level outcomes (ie, involving matched patient episode data) were assessed using multilevel logistics (eg, reattendance) or linear (costs) regression (random intercepts for doctors). Outcomes were regressed on doctor-level UT and adjusted for patient/doctor-level variables:
Model 1. Doctor UT + patient site
Model 2. Above + patient American Society of Anaesthesiologists (ASA) score.
Model 3. Above + patient age + patient gender
Model 4. Above + doctor experience (years) + doctor gender
Between-model fits were compared via Bayesian/Akaike Information Criteria. The within-model fit was assessed using R2 or pseudo-R2 metrics. Intraclass correlation coefficients (ICC) quantified between-doctor outcome variance.
It was hypothesised that higher UT would be associated with lower resource use, thus one-tailed tests were used. Two-tailed tests were planned for readmission, reattendance and mortality as it was hypothesised that UT could be associated in either direction. 95% CIs are provided for all estimates.
Missing data were minimal for most variables. Triage scores (acuity) had high missingness (eg, National Early Warning Scores (NEWS)) so ASA physical status scores were used for covariate adjustment instead. Sensitivity analyses assessed multiply-imputed acuity scores using all variables included in adjusted models as covariates. Other analyses assessed the removal of multivariate outliers and associations with specific versus composite indicators (eg, ECGs only vs all tests).
Patient and public involvement
The research team held regular meetings with the Sheffield Emergency Care Forum and Yorkshire & Humber Patient Safety Translational Research Centres (PSTRC) patient panel. These groups were updated on the study progress and were provided feedback on the direction, management and findings throughout.
Results
Recruitment and sample characteristics
41 doctors participated and matched patient episodes were obtained for 39 of these doctors (78% of the target 50) (table 1). The mean doctor age was 35.3 and a mean of 8.3 years working in EDs; 63.4% were women. Among all respondents, 12.5% reported having ‘no’ burnout.
Doctor and patient episode characteristics
Data for 384 patients were extracted and matched to the doctors (median=10, range=1–22 patients each). 54.7% of matched patients were women and their mean age was 46.6. The most prominent problems were chest (42.2%) and abdominal (25.8%) pain. Across episodes, doctors admitted 26% of patients. Tests ordered most were X-rays (29.4% episodes) and bloods (21.9%), the least MRI (0.3%) scans. 16% of patients reattended within 30 days of initial attendance, 7.6% were readmitted.
UT and doctor factors
Doctors’ mean UT score was 3.6/5 (range=2.6–4.8). The measure showed excellent internal reliability (α=0.92/ω=0.9224) indicating items consistently assessed one construct. Scores had low skewness (−0.01) and kurtosis (1.01) suggesting scores were evenly distributed around the mean with no extreme outliers or a sharp peak.
UT associations with categorical variables (eg, burnout) are shown in figure 1 and with continuous variables (eg, age in figure 2). Individuals with no burnout reported significantly higher tolerance compared with those with some (mean diff.=0.53; Cohen’s d=1.53; 95% CI=0.23 to 2.83) and moderate/high (collapsed categories) burnout (mean diff.=1.03; Cohen’s d=2.98; 95% CI=1.33 to 4.62). In turn, those with some reported significantly higher tolerance than those with moderate/high burnout (mean diff.=0.50; Cohen’s d=1.45; 95% CI=0.39 to 2.51).
Jitter plots, box plots and density plots for associations between uncertainty tolerance (UT) (higher=more tolerant) and all categorical variables: burnout, experience of an adverse event, site, grade and gender.
Histograms and density plots for all continuous variables plus scatterplots (with Pearson’s r estimates) between uncertainty tolerance (UT) (higher=more tolerant) and continuous variables: res (resilience), work (work life well-being), risk (risk aversion), hours (hours worked weekly), exp (years of experience), conf (confidence) and age. Could this be n (%)—clearer to readers fair point but it cannot be as the column sometimes references means and SD (eg, for age).
Higher resilience was also strongly associated with higher UT (Pearson’s r=0.56; 95% CI=0.30 to 0.74) as was higher work-life well-being (0.43; 0.13 to 0.65). Though higher confidence correlated with higher UT (0.27; −0.04 to 0.54), this was non-significant (p=0.09). There were significant UT differences between sites.
Experience of adverse events, hours worked, gender and seniority was not notably associated with UT.
UT and patient-level outcomes
Table 2 provides patient-level outcomes results. Point estimates indicated that as UT increased, episode costs decreased (eg, in Model IV looking at total costs shows there was a 7% decrease in costs per one UT scale point increase though we could not statistically rule out estimated increases in costs (0.07, 95% CI: −0.32 to 0.18). ICC was generally low (meaning there was low between-doctor variation in this outcome) and models explained small-to-moderate variance.
Uncertainty tolerance (UT) associations with patient-level outcomes
Tolerance was not significantly associated with the composite outcomes of rates of ordering ‘any test’, ‘any treatment’ or ‘any or test or treatment’. Models here explained small amounts of variance and ICC was low. UT was also not significantly associated with the proportion of patients with≤24-hour hospital stay after admission (extremely high R2 values indicated overfitting, meaning estimates should be interpreted cautiously). When the composite primary outcome of ‘any tests ordered’ was broken down into singular test outcomes (eg, CT scan: yes vs no), higher UT was significantly associated with less ECG ordering—but no other tests (see online supplemental file 6). Analyses showed a small decrease in reattendance and readmission rates with increased UT but estimates were again not significant. Models had low to moderate R2 values meaning other important variables were unaccounted for.
Supplemental material
Coefficients were practically equivalent when NEWS2 values were imputed and used in place of ASA scores. Removing multivariate outliers did not produce notable changes.
Discussion
Our UT measure demonstrated a good range and balance of scores, high internal consistency and significant correlations with self-reported doctor characteristics such as higher psychological resilience and lower burnout. There was no strong evidence of an association between tolerance and resource use; results indicated no significant associations and while coefficients mostly suggested an increase in costs with increased tolerance, CIs showed results consistent with the opposite. This pattern was also seen in patient outcomes.
While our UT measure addresses limitations with the PRUS including making it relevant to the UK emergency medicine context, it remains to be seen how the measure performs in further psychometric testing—for example, test–retesting—in larger samples. The measure appeared to be internally consistent and alpha/omega values were so high that some items may have been redundant and less burdensome measures could be derived. Future work within our team aims to conduct a more thorough validity assessment of our measure. We welcome future studies to adapt and assess the measure for UT assessment in other medical specialties and see whether scale-assessed tolerance is stable, modifiable and predictive of change in future behaviour.
The lack of significant associations between UT and resource use mirror findings from previous emergency medicine research,13–17 though not all specialties.3 One reason could be that patient pathways in emergency medicine are often protocolised, making significant practice variance difficult to capture. Homogenising the sample to ST3+ (ie, resident+level) doctors may have further reduced variation. Nonetheless, UT was linked with increased staff well-being suggesting it could be associated with resource use outcomes not captured here such as fewer staff sick days (eg, from burnout) and (eg, stress-induced) clinical errors.
Additional complexities in interpreting ED-based studies of UT arise from overlapping staff dynamics. In secondary analyses, we found that UT is significantly associated with lower ECG ordering rates but such relationships may be challenging to assess where several staff groups such as both nurses and doctors are empowered to order the same tests. Future studies could disentangle these subtleties, perhaps through path models and qualitative work.
For return admission outcomes, though small, regression coefficients indicated lower return rates with higher UT although it with CIs consistent with higher return rates. Thus, we could at least provide some evidence against positive associations of a medium to large size (ie, ORs above the upper estimate of our intervals which rarely exceeded 2). Future research needs to replicate our work with larger sample sizes to assess whether there are other outcomes not captured here that may be more sensitive or further contextual or other doctor factors which may moderate or mediate associations. For example, it could be that because the patients evaluated in this study all had high ‘uncertainty’ problems there was a ceiling effect; with more variation in problems, perhaps there would be more variation in decision-making processes sensitive to UT.
Consistent with previous studies,1 9 the differences in UT across sites suggest that it may be influenced by contextual factors. Future research could identify and investigate departments where doctors exhibit relatively higher average UT, providing insights into contributing factors that could be applied in other organisations25 to boost work-related well-being.
While we are unaware of trials of interventions to improve UT for doctors, recent research has identified uncertainty management strategies emergency doctors use already, providing a basis for intervention development.10–12 26 Patel demonstrated how visual thinking strategies and interactions with horses can prepare medical students to manage uncertainty by exposing them to a variety of clinical and non-clinical scenarios.27 Ilgen et al also highlighted that UT may manifest not only as a personality trait but also as a situational response that varies with clinical context.6 These insights suggest a need for interventions that are both adaptive and context-specific using real-world scenarios to effectively cultivate tolerance among healthcare staff.
Limitations
Records data were taken retrospectively, so some doctors may have deferred decisions to colleagues without noting it, undermining the assumption that costs resulted from their decisions. Similarly, UT scores may not reflect past scores if tolerance-modifying events occurred between episodes and doctor participation.
Because the coefficients in our analysis cannot be interpreted causally, future studies should consider modifying UT through randomised trials to assess causal effects. We did not meet our sample size targets due to overestimating achievable recruitment rates; as a result, our CIs were wider and less precise, making it difficult to detect smaller effects suggested by our point estimates. Convenience sampling may have biased our doctor sample towards those who are more confident and relying on self-reported data could have introduced social desirability bias.28 Additionally, the self-selection of sites may have led to the inclusion of locations with specific characteristics such as particular organisational cultures.
While no associations between UT and negative patient outcomes were found, the proxy measures chosen may have been too insensitive. Future research should capture broader outcomes such as 30-day patient general practitioner visits and provide more granular analyses of how UT relates to outcomes in specific contexts. For example, research could identify patient groups that are more often inappropriately dismissed or overlooked by doctors with higher UT and assess how this more targeted issue could be mitigated.
Studies have suggested UT can only be investigated as ‘state’11 12 but this is disputed1 10 not least because UT may be stable over time.29 Further, some argue neither extreme tolerance nor intolerance may be inherently beneficial or detrimental as each could provide advantages and expose clinicians to risks in practice, undermining our assumptions that higher scores were ‘better’.30
While costs were taken from tariffs, some potentially unrealistic assumptions were made, for example, assuming a medication packet was opened for each given dose and some costs were missed such as tests on reattendance.
Some measures used only one question for example, risk aversion and burnout thus not very sensitive to the complexity of these concepts.
Conclusion
Higher doctor UT on a novel measure was associated with higher self-reported psychological and work-related well-being but was not significantly associated with patient outcomes or most measures of resource use. Interventions to support emergency doctors to better tolerate uncertainty might have positive implications for them but our study is unable to answer the question as to whether there are positive or negative side effects on patient outcomes. It is necessary for future work to replicate and build on our study to resolve such ambiguities.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Our protocol and materials (accessible here: github.com/Budlytics/uncertainty_tolerance) were approved by a Health Research Authority ethics committee (IRAS ID: 202830; East of Scotland REC).
Acknowledgments
Thank you, Professor Tracy Young, for your advice and our data extractors, Amy Greaves, Kiran Bhangu, Shreya Agrawal, Alisa Morris, Hannah Wakefield and Jonathon Heron.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Bhakti Hansoti
X @lukewbudworth, @ProfSueMason
Contributors LB, BW, SBas, CO’K, SMM, SA-W, SC, MC, RL designed the study, including selecting questionnaire measures, developing our uncertainty tolerance measure and deciding on outcome measures. LB managed the data sets, designed the statistical analysis plan and analysed the study data with support and supervision from ADS. BW, JS-K, SBas, SMM, AA, SA-W, KR, SC all extracted data from patients medical records and contributed to the development of the patient data extraction tool in a pilot. BW, JS-K, SBas, SMM, AA, SA-W, KR, SC approached and recruited colleagues into the study to complete our doctor questionnaire. SBan extracted cost values from online resources. LB, BW, JS-K, SBas, CO’K, SMM, AA, SA-W, KR, SC, ADS, SBan, MC, RL all contributed to both the writing and editing of the manuscript. LB is the guarantor of the study.
Funding This report is independent research funded by the National Institute for Health and Care Research Yorkshire and Humber Applied Research Collaboration (ARC) (NIHR200166) and the National Institute for Health and Care Research Yorkshire and Humber Patient Safety Research Collaboration (PSRC). The views expressed in this publication are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care. ADS is funded by a postdoctoral fellowship from THIS Institute, NIHR (AI_AWARD01864 and COV-LT-0009), UKRI (Horizon Europe Guarantee for DataTools4Heart) and British Heart Foundation Accelerator Award (AA/18/6/24223).
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.