Chapter 3 explains the research method being used in the

March 2023 1

DSE: Chapter 3 – Evaluation Rubric

Criteria Does Not Meet 0.01 points Meets 1 point

Introductory Remarks

The section is missing; or some topic areas are not included in the Introduction or are not explained clearly. The chapter outline is not provided and/or is unclear.

The reader is adequately oriented to the topic areas covered. An outline of the logical flow of the chapter is presented. All major themes/concepts are introduced.

Research Methodology and Design

There is a lack of alignment among the chosen research method and design and the study’s problem, purpose, and research questions. There is a lack of justification and alternate choices for methods. For Qualitative Studies: Lacks clear discussion of the study phenomenon, boundaries of case(s), and/or constructs explored.

Describes how the research method and design are aligned with the study problem, purpose, and research questions. Uses scholarly support to describe how the design choice is consistent with the research method, and alternate choices are discussed. For Qualitative Studies: Describes the study phenomenon, boundaries of case(s) and/or constructs explored.

Population and Sample

Lacks a description of the instruments associated with the chosen research method and design used. Details missing regarding instrument origin, reliability, and validity. For Quantitative Studies: (e.g., tests or surveys). Lacks explanation of any permission needed to use the instrument(s) and cites properly. Instrument permissions are missing in appendices

Provides a description of the instruments associated with the chosen research method and design used. Includes information regarding instrument origin, reliability, and validity. For Quantitative Studies: (e.g., tests or surveys). Includes any permission needed to use the instrument(s) and cites properly.

March 2023 2

DSE: Chapter 3 – Evaluation Rubric

For Qualitative Studies: (e.g., observation checklists/protocols, interview or focus group discussion Handbooks). Did not clearly explain the process for conducting an expert review of instruments (e.g., provides justification of reviewers being credible – reviewers may include, but not limited to NCU dissertation team members, professional colleagues, peers, or non-research participants representative of the greater population); and/or did not clearly explain use of a field test if practicing the administration of the instruments is warranted. For Pilot Study: Does not clearly explain the procedure for conducting a pilot study (did not conduct pilot) if using a self-created instrument (e.g., survey questionnaire); does not include explanation of a field test if practicing the administration of the instruments is warranted.

For Qualitative Studies: (e.g., observation checklists/protocols, interview or focus group discussion Handbooks). Describes process for conducting an expert review of instruments (e.g., provides justification of reviewers being credible – reviewers may include, but not limited to NCU dissertation team members, professional colleagues, peers, or non-research participants representative of the greater population); describes use of a field test if practicing the administration of the instruments is warranted. For Pilot Study: Explains the procedure for conducting a pilot study (requires IRB approval for pilot) if using a self-created instrument (e.g., survey questionnaire); explains use of a field test if practicing the administration of the instruments is warranted.

Operational Definitions of Variables (Quantitative Studies Only)

Discussion of the study variables examined is lacking information and/or is unclear.

Describes study variables in terms of being measurable and/or observable. (Reviewer – mark Meets/NA for Qualitative studies)

Procedures Procedures are not clear or replicable. Steps are missing; recruitment, selection, and informed consent are not established. IRB ethical practices are missing or unclear.

Describes the procedures to conduct the study in enough detail to practically replicate the study, including participant recruitment and notification, and informed consent. IRB ethical practices are noted.

March 2023 3

DSE: Chapter 3 – Evaluation Rubric

Data Collection and Analysis

Does not clearly provide a description of the data and the processes to collect data. Lack of alignment between the data collected and the research questions and/or hypotheses of the study. For Quantitative Studies: Does not clearly provide the data analysis processes including, but not limited to, clearly describing the statistical tests performed and the purpose/outcome, coding of data linked to each RQ, the software used (e.g., SPSS, Qualtrics). For Qualitative Studies: Does not clearly identify the coding process of data linked to RQs; does not clearly describe the transcription of data, the software used for textual analysis (e.g., Nivo, DeDoose), and does not justify manual analysis by researcher. There is missing or unclear explanation of the use of a member check to validate data collected.

Provides a description of the data collected and the processes used in gathering the data. Explains alignment between the data collected and the research questions and/or hypotheses of the study. For Quantitative Studies: Includes the data analysis processes including, but not limited to, describing the statistical tests performed and the purpose/outcome, coding of data linked to each RQ, the software used (e.g., SPSS, Qualtrics). For Qualitative Studies: Identifies the coding process of data linked to RQs. Describes the transcription of data, the software used for textual analysis (e.g., Nivo, DeDoose), and describes manual analysis by researcher. Describes the use of a member check to validate data collected.

Assumptions/ Limitations/ Delimitations

Does not clearly outline the assumptions/limitations/ delimitations (or has missing components) inherent to the choice of method and design. For Quantitative Studies: Does not include or lacks key elements such as, but not limited to, threats to internal and external validity, credibility, and generalizability. For Qualitative Studies: Does not include or lacks key elements such as, but not limited to threats to credibility, trustworthiness, and transferability.

Outlines the assumptions/limitations/delimitations to the choice of method and design. For Quantitative Studies: Includes key elements such as, but not limited to, threats to internal and external validity, credibility, and generalizability. For Qualitative Studies: Includes key elements such as, but not limited to threats to credibility, trustworthiness, and transferability.

March 2023 4

DSE: Chapter 3 – Evaluation Rubric

Ethical Assurances

Lacks discussion of compliance with the standards to conduct research as appropriate to the proposed research design and is not aligned to IRB requirements.

Describes compliance with the standards to conduct research as appropriate to the proposed research design and aligned to IRB requirements.

Summary Chapter does not conclude with a summary of key points from the Chapter – elements are missing, incomplete, and/new information is presented.

Chapter concludes with an organized summary of key points discussed/presented in the Chapter.