Discuss the reasons for using mixed methods presented in the chapter. For each reason, provide an example of a study that would benefit. Discuss at least one example of integration in mi

There have been several typologies for classifying and identifying types of mixed methods strategies that proposal developers might use in their proposed mixed methods study. Creswell and Plano Clark (2018) identified several classification systems drawn from the fields of evaluation, nursing, public health, education policy and research, and social and behavioral research. In these classifications, authors used diverse terms for their types of designs, and a substantial amount of overlap of types existed in the typologies. For purposes of clarifying the design discussion in the mixed methods field, we will identify three core mixed methods designs (as shown in Figures 10.1 and 10.2)—the convergent design, the explanatory sequential design, and the exploratory sequential design—and then briefly mention more complex designs (i.e., the mixed methods experimental design, the mixed methods case study design, the mixed methods participatory–social justice design, and the mixed methods evaluation design) in which the core designs can be embedded. Each approach will be discussed in terms of a description of the design, the forms of data collection and data analysis and integration, interpretation, and validity challenges.

Convergent Mixed Methods Design

Description of the design. The convergent mixed methods design is probably the most familiar of the core and complex mixed methods approaches. Researchers new to mixed methods typically first think of this approach because they feel that mixed methods only consists of combining the quantitative and qualitative data. In this single-phase approach, a researcher collects both quantitative and qualitative data, analyzes them separately, and then compares the results to see if the findings confirm or disconfirm each other (see Figure 10.1). The key assumption of this approach is that both qualitative and quantitative data provide different types of information—often detailed views of participants qualitatively and scores on instruments quantitatively—and together they yield results that should be the same. It builds off the historic concept of the multimethod, multitrait idea from Campbell and Fiske (1959), who felt that a psychological trait could best be understood by gathering different forms of data. Although the Campbell and Fiske conceptualization included only quantitative data, the mixed methods researchers extended the idea to include the collection of both quantitative and qualitative data.

Figure 10.1 Three Core Mixed Methods Designs

Data collection. The qualitative data can assume any of the forms discussed in Chapter 9, such as interviews, observations, documents, and records. The quantitative data can be instrument data, observational checklists, or numeric records, such as census data, as discussed in Chapter 8. Ideally, the key idea with this design is to collect both forms of data using the same or parallel variables, constructs, or concepts. In other words, if the concept of self-esteem is being measured during quantitative data collection, the same concept is asked during the qualitative data collection process, such as in an open-ended interview. Some researchers will use this design to associate certain themes with statistical data using different forms of data for the quantitative and qualitative data collection. For instance, Shaw et al. (2013) compared quality improvement practices in family medicine clinics with colorectal cancer screening rates. Another data collection issue is the sample size for both the qualitative and quantitative data collection process. Unquestionably, the data for the qualitative data collection will be smaller than that for the quantitative data collection. This is because the intent of data collection for qualitative data is to locate and obtain information from a small sample but to gather extensive information from this sample; whereas, in quantitative research, a large N is needed in order to infer meaningful statistical results from samples to a population.

How is this inequality resolved in a convergent mixed methods design? Sometimes mixed methods researchers will collect information from the same number of individuals on both the qualitative and quantitative databases. This means that the qualitative sample will be increased, and it will limit the amount of data collected from any one individual. Another approach would be to weight the qualitative cases so that they equal the N in the quantitative database. One other approach taken by some mixed methods researchers is not to consider the unequal sample sizes a problem. They would argue that the intent of qualitative and quantitative research differs (one to gain an in-depth perspective and the other, to generalize to a population) and that each provides an adequate count. Another issue in sampling is whether the individuals for the sample of qualitative participants should also be individuals in the quantitative sample. Typically, mixed methods researchers would include the sample of qualitative participants in the larger quantitative sample, because ultimately researchers make a comparison between the two databases and the more they are similar, the better the comparison.

Data analysis and integration. Data analysis in a convergent design consists of three phases. First, analyze the qualitative database by coding the data and collapsing the codes into broad themes. Second, analyze the quantitative database in terms of statistical results. Third comes the mixed methods data analysis. This is the analysis that consists of integrating the two databases.

This integration consists of merging the results from both the qualitative and the quantitative findings. One challenge in this design is how to actually merge the two databases since bringing together a numeric quantitative database with a text qualitative database is not intuitive. There are several ways to merge the two databases:

The first approach is called a side-by-side comparison. These comparisons can be seen in the discussion sections of mixed methods studies. The researcher will first report the quantitative statistical results and then discuss the qualitative findings (e.g., themes) that either confirm or disconfirm the statistical results. Alternatively, the researcher might start with the qualitative findings and then compare them to the quantitative results. Mixed methods writers call this a side-by-side approach because the researcher makes the comparison within a discussion, presenting first one set of findings and then the other. A good example of this can be seen in the Classen and colleagues’ (2007) study.

Researchers can also merge the two databases by changing or transforming qualitative codes or themes into quantitative variables and then combining the two quantitative databases—a procedure in mixed methods research called data transformation. The researcher takes the qualitative themes or codes and counts them (and possibly groups them) to form quantitative measures. Some useful procedures that mixed methods researchers have used can be found in Onwuegbuzie and Leech (2006). This approach is popular among researchers trained in quantitative research who may not value or see the worth of an independent qualitative interpretive database.

A final procedure involves merging the two forms of data in a table or a graph. This is called a joint display of data, and it can take many different forms. It might be a table that arrays the themes on the horizontal axis and a categorical variable (e.g., different types of providers such as nurses, physician assistants, and doctors) on the vertical axis. It might be a table with key questions or concepts on the vertical axis and then two columns on the horizontal axis indicating qualitative responses and quantitative responses to the key questions or concepts (Li, Marquart, & Zercher, 2000). The basic idea is for the researcher to jointly display both forms of data—effectively merging them—in a single visual and then make an interpretation of the display (see Guetterman, Fetters, & Creswell, 2015).

Interpretation. The interpretation in the convergent approach is typically written into a discussion section of the study. Whereas the results section reports on the findings from the analysis of both the quantitative and qualitative databases, the discussion section includes a discussion comparing the results from the two databases and notes whether there is convergence or divergence between the two sources of information. Typically the comparison does not yield a clean convergent or divergent situation, and differences exist on a few concepts, themes, or scales. When divergence occurs, steps for follow-up need to be taken. The researcher can state divergence as a limitation in the study without further follow-up. This approach represents a weak solution. Alternatively, mixed methods researchers can return to the analyses and further explore the databases, collect additional information to resolve the differences, or discuss the results from one of the databases as possibly limited (e.g., the constructs were not valid quantitatively or the qualitative themes did not match the open-ended questions). Whatever approach the researcher takes, the key point in a convergent design is to further discuss and probe results when divergent findings exist.

Validity. Validity using the convergent approach should be based on establishing both quantitative validity (e.g., construct) and qualitative validity (e.g., triangulation) for each database. Is there a special form of mixed methods validity that needs to be addressed? There are certainly some potential threats to validity in using the convergent approach, and several of these have already been mentioned. Unequal sample sizes may provide less of a picture on the qualitative side than the larger N on the quantitative side. Generally we find the use of unequal sample sizes in a convergent design study, with the researcher acknowledging the different perspectives on size taken by quantitative and qualitative researchers. The use of different concepts or variables on both sides, quantitative and qualitative, may yield incomparable and difficult-to-merge findings. Our recommended approach is to use the same concepts for both the quantitative and qualitative arms of the research study, but we acknowledge that some researchers use the convergent design to associate different qualitative and quantitative concepts. A lack of follow-up on conclusions when the scores and themes diverge also represents an invalid strategy of inquiry. In this discussion we have recommended several ways to probe divergence in more detail and would recommend the use of one or more of these strategies in a convergent design project.

Explanatory Sequential Mixed Methods Design

Description of the design. The explanatory sequential mixed methods approach is a design in mixed methods that appeals to individuals with a strong quantitative background or from fields relatively new to qualitative approaches. It involves a two-phase data collection project in which the researcher collects quantitative data in the first phase, analyzes the results, and then uses the results to plan (or build on to) the second, qualitative phase. The quantitative results typically inform the types of participants to be purposefully selected for the qualitative phase and the types of questions that will be asked of the participants. The overall intent of this design is to have the qualitative data help explain in more detail the initial quantitative results, thus it is important to tie together or to connect the quantitative results to the qualitative data collection. A typical procedure might involve collecting survey data in the first phase, analyzing the data, and then following up with qualitative interviews to help explain confusing, contradictory, or unusual survey responses.

Data collection. The data collection proceeds in two distinct phases with rigorous quantitative sampling in the first phase and with purposeful sampling in the second, qualitative phase. One challenge in this strategy is to plan adequately what quantitative results to follow up on and what participants to gather qualitative data from in the second phase. The key idea is that the qualitative data collection builds directly on the quantitative results. The quantitative results that then are built on may be extreme or outlier cases, significant predictors, significant results relating variables, insignificant results, or even demographics. For example, when using demographics, the researcher could find in the initial quantitative phase that individuals in different socioeconomic levels respond differently to the dependent variables. Thus, the follow-up qualitatively may group respondents to the quantitative phase into different categories and conduct qualitative data collection with individuals representing each of the categories. Another challenge is whether the qualitative sample should be individuals that are in the initial quantitative sample. The answer to this question should be that they are the same individuals, because the intent of the design is to follow up the quantitative results and explore the results in more depth. The idea of explaining the mechanism—how the variables interact—in more depth through the qualitative follow-up is a key strength of this design.

Data analysis and integration. The quantitative and the qualitative databases are analyzed separately in this approach. Then the researcher combines the two databases by the form of integration called connecting the quantitative results to the qualitative data collection. This is the point of integration in an explanatory sequential design. Thus, the quantitative results are then used to plan the qualitative follow-up. One important area is that the quantitative results cannot only inform the sampling procedure but it can also point toward the types of qualitative questions to ask participants in the second phase. These questions, like all good qualitative research questions, are general and open-ended. Because analysis proceeds independently for each phase, this design is useful for student research and perhaps easier to accomplish (than the convergent design) because one database explains the other and the data collection can be spaced out over time.

Interpretation. The mixed methods researcher interprets the follow up results in a discussion section of the study. This interpretation follows the form of first reporting the quantitative, first-phase results and then the qualitative, second phase results. However, this design then employs a third form of interpretation: how the qualitative findings help to explain the quantitative results. A common misstep at this point by beginning researchers is to merge the two databases. While this approach may be helpful, the intent of the design is to have the qualitative data help to provide more depth, more insight into the quantitative results. Accordingly, in the interpretation section, after the researcher presents the general quantitative and then qualitative results, a discussion should follow that specifies how the qualitative results help to expand or explain the quantitative results. Because the qualitative database questions narrows the scope of the quantitative questions, a direct comparison of the overall results of the two databases is not recommended.

Validity. As with all mixed methods studies, the researcher needs to establish the validity of the scores from the quantitative measures and to discuss the validity of the qualitative findings. In the explanatory sequential mixed methods approach, additional validity concerns arise. The accuracy of the overall findings may be compromised because the researcher does not consider and weigh all of the options for following up on the quantitative results. We recommend that researchers consider all options for identifying results to follow up on before settling on one approach. Attention may focus only on personal demographics and overlook important explanations that need further understanding. The researcher may also contribute to invalidated results by drawing on different samples for each phase of the study. If explaining the quantitative results in more depth, then it makes sense to select the qualitative sample from individuals who participated in the quantitative sample. This maximizes the importance of one phase explaining the other. These are a few of the challenges that need to be built into the planning process for a good explanatory sequential mixed methods study.

Exploratory Sequential Mixed Methods Design

Description of the design. If we reverse the explanatory sequential approach and start with a qualitative phase first followed by a quantitative phase, we have an exploratory sequential approach. A three-phase exploratory sequential mixed methods is a design in which the researcher first begins by exploring with qualitative data and analysis, then builds a feature to be tested (e.g., a new survey instrument, experimental procedures, a website, or new variables) and tests this feature in a quantitative third phase. Like the explanatory sequential approach, the second feature builds on the results of the initial database. The intent of this design is to explore with a sample first so that a later quantitative phase can be tailored to meet the needs of the individuals being studied. Sometimes this quantitative feature will include developing a contextually sensitive measurement instrument and then testing it with a sample. Other times it may involve developing new variables not available in the literature or attuned to a specific population being studied, or designing a website or an Internet application shaped to the needs of the individuals being studied. This design is popular in global health research when, for example, investigators need to understand a community or population before administering English-language instruments.

In this design, the researcher would first collect focus group data, analyze the results, develop an instrument (or other quantitative feature such as a website for testing), and then administer it to a sample of a population. In this case, there may not be adequate instruments to measure the concepts with the sample the investigator wishes to study. In effect, the researcher employs a three-phase procedure with the first phase as exploratory, the second as instrument (or quantitative feature) development, and the third as administering and testing the instrument feature to a sample of a population.

Data collection. In this strategy, the data collection would occur at two points in the design: the initial qualitative data collection and the test of the quantitative feature in the third phase of the project. The challenge is how to use the information from the initial qualitative phase to build or identify the quantitative feature in the second phase. This is the integration point in an exploratory sequential design.

Several options exist, and we will use the approach of developing a culturally sensitive instrument as an illustration. The qualitative data analysis can be used to develop an instrument with good psychometric properties (i.e., validity, reliability). The qualitative data analysis will yield quotes, codes, and themes (see Chapter 9). The development of an instrument can proceed by using the quotes to write items for an instrument, the codes to develop variables that group the items, and themes that group the codes into scales. This is a useful procedure for moving from qualitative data analysis to scale development (the quantitative feature developed in the second phase). Scale development also needs to follow good procedures for instrument design, and the steps for this include ideas such as item discrimination, construct validity, and reliability estimates (see DeVellis, 2012).

Developing a good psychometric instrument that fits the sample and population under study is not the only use of this design. A researcher can analyze the qualitative data to develop new variables that may not be present in the literature, to identify the types of scales that might exist in current instruments or to form categories of information that will be explored further in a quantitative phase. The question arises if the sample for the qualitative phase is the same for the quantitative phase. This cannot be, because the qualitative sample is typically much smaller than a quantitative sample needed to generalize from a sample to a population. Sometimes mixed methods researchers will use entirely different samples for the qualitative (first phase) and quantitative components (third phase) of the study. However, a good procedure is to draw both samples from the same population but make sure that the individuals for both samples are not the same. To have individuals help develop an instrument and then to survey them in the quantitative phase would introduce confounding factors into the study.

Data analysis and integration. In this strategy the researcher analyzes the two datab