This paper follows previous publications on generic qualitative approaches, qualitative designs and action research in emergency care by this group of authors. Contemporary views on mixed methods approaches are considered, with a particular focus on the design choice and the amalgamation of qualitative and quantitative data emphasising the timing of data collection for each approach, their relative ‘weight’ and how they will be mixed. Mixed methods studies in emergency care are reviewed before the variety of methodological approaches and best practice considerations are presented. The use of mixed methods in clinical studies is increasing, aiming to answer questions such as ‘how many’ and ‘why’ in the same study, and as such are an important and useful approach to many key questions in emergency care.
- mixed methods
- clinical assessment
- emergency care systems
- emergency departments
Statistics from Altmetric.com
- mixed methods
- clinical assessment
- emergency care systems
- emergency departments
In our previous work we have described generic (or descriptive) approaches to qualitative research,1 the more focused qualitative approaches of grounded theory, phenomenology and ethnography2; and the pragmatic problem resolution approaches used in action research/learning.3 While not without its critics,4 5 mixed method approaches, using qualitative and quantitative approaches in the same study,6 are commonly adopted by pragmatists7 who employ diverse approaches ‘that work’, while valuing subjective and objective knowledge. The resultant data sets produce a better understanding of the phenomenon being studied, with each paradigm extending the researcher's view and enabling complex human phenomena to be captured.8 The positivist approach produces a singular and objective view of the world, by predicting and testing relationships, while an interpretative approach explores and generates meaning within the practice context.9 The combination, therefore enriches and enhances the findings10 and may answer many of the key questions in clinical settings such as emergency care. For example reasons for patients visiting the emergency department (ED)11–13 and factors influencing waiting times14 have been the subject of many studies. While analysis of quantitative data can identify links between variables such as dissatisfaction with primary care and non-urgent visits to the ED,11 the addition of a qualitative method with the same population could also be used to examine how changes in service provision might influence behaviour.
Previous examination of missed diagnosis in ED has focused on quantifying the impact of factors such as gender, ethnic background and age.15 A mixed methods study with a sample of clinicians drawn from the same health service could examine why these factors have an impact on diagnosis decisions.
Mixed methods approaches are also of value in patient safety research as they allow researchers to examine why an intervention did or did not work16, while multiple end points at various stages along the ‘causal chain’ can be studied.17 The question of ‘what is truth’ is answered in mixed methods research by taking a pragmatic approach, described by Brown and colleagues as follows: ‘a position whereby strength of belief accumulates in line with salient evidence’.18
A background literature review was performed based upon a search of Medline, the Cumulative Index to Nursing and Allied Health Literature (CINHAL) and Google Scholar for papers relating to a mixed methods approach in the emergency care field. In all, 10 papers were identified that had an emergency care focus with a mixed methods approach; these are summarised in table 1.
It is likely that many other studies have used mixed methods, however the identification of such is difficult, as papers often do not clearly state that they have used a mixed method or multimethod approach. However, from this diverse set of studies, it is clear that mixed methods does have a place in the emergency care research field and that they do enhance our understanding of practice. In the following we summarise the central tenets of a mixed method approach considering the design and rigour of the approach.
Mixed methods designs
In this section we focus on the design choice and the amalgamation of qualitative and quantitative data focusing on the timing of data collection for each approach, their relative ‘weight’ and when and how data will be mixed. Research problems should be matched to applicable methods and designs, for example experimental designs to test the effectiveness of a treatment, correlational to identify factors that influence an outcome, ethnographic to describe a culture and grounded theory to generate a theory.6 Mixed methods tend to be used when there is a need to enhance and compliment a study with a second data source.28 for example following a survey with interviews for a more in-depth understanding; qualitative data to help explain a correlation; quantitative data to support qualitative (eg, measuring frequency counts in interviews and observations); and qualitative approaches for a greater understanding of a topic prior to an experiment.
Creswell and Plano Clark6 summarise the four core design choices as follows.
Within this choice are four models6:
The convergent model: quantitative and qualitative data are collected and analysed separately with comparison of the two sets of results at the end of the study, aiming to produce a final amalgamated interpretation. For example, hospital episode data may be reviewed to provide a profile of patients who typically leave the ED before treatment; follow-up interviews with the patients may examine why they left ED, what action they took subsequently and what type of access they have to primary care services. These two sets of findings can be amalgamated at the conclusion of the study to provide a more complete picture.
The data transformation model: quantitative and qualitative data are collected separately but the qualitative data transformed into quantitative prior to final analysis, a process also known as ‘quantitising’,8 for example the transformation of qualitative observation data into summary counts of the number of health professionals a patient sees on their journey through the ED.
The validating quantitative model: qualitative data are collected to support a quantitative data set, for example survey data may include open, free text questions, the responses from which are then used to validate the quantitative findings.
The multilevel model: quantitative and qualitative data are collected at different levels prior to final amalgamation; for example studying patient flow through the ED by interviewing nurses and doctors (qualitative), conducting focus groups for managers (qualitative) and analysing organisational data (quantitative) to identify bottlenecks in patient flow.3
In the triangulation design the timing of the data collection is usually concurrent; the weighting of quantitative and qualitative data is usually equal; and the mixing of data is usually performed during the analysis and interpretation stage. Reporting practice tends to include separate quantitative and qualitative reports that are merged in the results or discussion section.
This is an approach that embeds a secondary (supplementary) form of data within the primary source, for example collecting qualitative data to inform development of a treatments (eg, observation or interviews), followed by a quantitative pretest, the treatment intervention, quantitative post-test and a qualitative follow through to aid interpretation of a predominantly quantitative approach. The timing of data collection is sequential or concurrent; the weighting is unequal for example quantitative is given greater prominence than qualitative; and the mixing of data occurs throughout the data collection phase informing the development of the study and the results. Again reporting practice tends to include separate quantitative and qualitative reports that are merged in the results or discussion section.6
This is a two phase approach using qualitative data (phase 2) to explain quantitative data (phase 1), for example examining records of triage activity using a scoring system followed be observation and informal interviews, recording field notes to explain how and why triage decisions are made. The timing of data collection is quantitative followed by qualitative; the weighting is predominantly quantitative; and the mixing of data, follows analysis of quantitative data leading to collection and analysis of qualitative data. Reporting methods tend to be based on a sequential reporting of results.6
Again, this is a two-phase approach usually adopted when little is known about a phenomenon, for example when developing a new assessment instrument on emergency teamwork the researcher may observe and interview teams prior to developing a quantitative rating scale. The timing of data collection will be qualitative followed by quantitative; the weighting is usually qualitative; and the mixing of data, follows analysis of qualitative data leading to collection and analysis of quantitative data. Again results tend to be reported in sequence.6
The emergence of mixed methods as a research design with specific rules is relatively recent; a number of earlier studies use the term mixed methods to simply describe inclusion of more than one method in a single study but without making decisions about weight, timing and mixing of data explicit.
For any of the designs it is important to carefully consider the mixed methods questions that are being addressed, for example in a triangulated convergent design it will be ‘how do the results converge?’, and for an embedded design it will be ‘how do the qualitative results explain or expand on the quantitative results?’. Sampling procedures29 should, in general, be adequate for each quantitative and qualitative approach and for the design selected; for example in an embedded design the embedded data is likely to be derived from a smaller sample. Data analysis will be dependant on the design selected, for example following the completion of data collection for a triangulated convergent model, or after each stage for an explanatory design. Where data is transformed qualitative themes may be counted or presented in the form of a metasummary,30 discussion, or matrix relating to quantitative counts and qualitative themes.6 In addition mixed methods research is particularly suited to statistical models such as the Bayesian approach.17
One essential consideration for the mixed methods researcher is the resolution of contradictory results from the quantitative and qualitative data? Several approaches are advised:
more data can be collected in the hope of resolving the contradiction,
re-examination of the data can be performed, or
For example, in a mixed methods study Moffatt et al32 identified conflicting results between the data sets when exploring the impact of welfare rights advice in primary care. They therefore explored the data further by, examining the rigour of each component; exploring the comparability of the data sets; and collecting additional data.
Researchers should also be aware that results may be confounded where data is collected concurrently from the same participants, for example interviews before a quantitative trial. Less obtrusive methods such as broadly focused respondent diaries, or equal distribution of data collection between control and experimental groups,6 may resolve the issue. The validity and reliability (rigour) of the data is confirmed by the standard method for each approach,1 2 for example with sample power calculations and aiming for thematic ‘saturation’. However, Creswell and Plano Clark6 also advise that validity and reliability are discussed (and the threats made clear) within the context of the mixed approach, with validity defined in the mixed methods context as the ‘ability of the researcher to draw meaningful and accurate conclusions from all the data in the study’.
When writing reports it is important to clearly state the problem and the research question/hypotheses, and also describe the design, timing, weighting and mixing, and the rationale for the design selected.6 28 In our above review of mixed methods studies in emergency care this was never the case; reports tended to introduce the studies as being mixed methods and describing the individual methods but without any clarity on the design and analytical approach.
Mixed methods are often ideal for the pragmatic clinical researcher, especially when a transformational and ‘bedside’ research approach is required. An eclectic mix of studies has been identified in the emergency care arena that illustrate the benefits of the approach. However, an understanding of the design options, timing, weighting and mixing are essential, with careful consideration given to the research questions, sampling, data analysis, validity and reliability, and resolution of conflicting results.
Research conducted in the emergency care setting evaluates a complexity of interventions that requires the use of quantitative and qualitative evidence.33 Unfortunately some of the prime research texts for undergraduate and postgraduate teaching do not include a mixed methods section,9 34 reducing the methodological choices available to health professionals.
Mixed methods approaches are not without their critics. It is argued4 that quantitative and qualitative approaches are derived from fundamentally different philosophical foundations and that the phenomena are quite different. There is a danger that researchers are failing to consider the underlying assumptions behind quantitative and qualitative approaches and that mixed methods approaches are being adopted uncritically. Sale et al,4 for example argue that the two approaches are essentially a study of different phenomena and that one should not be used to bolster the weakness of another, but incorporated to inform and add value as mutual research approaches. These criticisms emanate largely from experts in qualitative research methods who suggest that qualitative data is given lower weighting and priority in mixed methods research.5 Other critics35 suggest that using mixed methods runs the risk of not providing sufficient data for ‘cross-methods comparisons’. In our view, there is great value in using qualitative methods to enrich our understanding of emergency medicine, while a mixed approach is likely to add depth and breadth to the research and its findings.2
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.