Preparation Review the following theoretical study and program evaluation samples used in your Week 3 assignment before completing this assignment: Week

1

Applying Knowledge of Types of Research

Audrian Hammond

Professor Ferrer

Capella University

HMSV8008

July 25, 2025

2

Introduction

Structured inquiry directs evidence-based decision-making and informs interventions

that are significantly valuable to individuals and communities and is central to human services

practitioners and researchers. In this process, there are two major research styles, theoretical

research and program evaluation, that have complementary roles. Theoretical studies aim to

devise or implement a conceptual framework in a controlled environment, while program

evaluation measures the effectiveness and practical application of programs. Although these

methodologies share common results in empirical methods and ethical-protective considerations,

there are differences in terms of proposed objectives, methodology, actors, and the nature of the

data to be used. In this paper, I am analyzing two of your sources, which are either a theoretical

study or program evaluation, comparing their purpose according to their exact wording, research

questions, and type of research design, data sources, stakeholder involvement, and ethical and

legal considerations. Making this comparison, this paper will demonstrate that an in-depth

theoretical examination, as well as a practice-based analysis of the program, is essential for

creating credible and situation-specific human services.

Comparing Definitions, Objectives, and Scope of Inquiry.

Theoretical research, as in climatological research by Berhail and Katipoglu (2023),

aims to test or compare scientific instruments in a controlled environment, thereby contributing

to the general body of scientific knowledge. The authors of that study compared the performance

of the Standardized Precipitation Index (SPI) and the Standardized Precipitation

Evapotranspiration Index (SPEI) in drought evaluation over a semi-arid Algerian region. They

wanted to enhance the theoretical knowledge of the index that best defines variability drought

when stressed in climatic terms hence bettering the future use in climatology. Program

3

evaluation, in contradiction, centers on the impact of a policy or program when putting it into

practice, like economic ways of evaluation such as the difference-in-differences (DiD)

framework presented by Callaway, Goodman-Bacon and SantAnna (2024) and Gardner (2022).

DiD techniques, which are designed to infer the impact of issues in observational studies, can be

adapted to inform real-world policy assessments. Callaway and others generalized conventional

DiD analysis to support continuous values in treatment, which makes it more beneficial in policy

studies. Gardner then further developed the evaluation methodology by using a two-stage DiD

approach, which aims to facilitate causal interpretation in cases where there are multiple

treatment timings. In this manner, conceptual boundaries are advanced through theoretical

research, whereas program evaluation employs empirical forms of research that are grounded in

theory to address practical questions.

Purpose Statements and Research Questions

In the theoretical analysis conducted by Berhail and Katipoğlu (2023), the aim was clearly

stated: to compare SPI with SPEI in estimating drought intensity and length in a semi-arid

region, thereby enabling a more effective evaluation of drought in future climatology studies.

The first research question, which they considered, was as follows: “Which drought index (SPI

or SPEI) is more effective to gauge the features of droughts in the Wadi Mekerra basin, and

under which climatic and time conditions?” This was the same in exactly the kind of rigour and

specificity in which most theoretical research is conducted.

Conversely, Callaway, Goodman-Bacon, and Sant Anna (2024) stated in the purpose

that their study aimed to promote the evaluation methodology for addressing issues that exist in

contexts of continuous treatment. They aimed at identifying the correct ways of estimating causal

effects in situations where treatment strength differs across individuals and time, as is the case

4

with policy implementation. Their research question was, “What is the most appropriate form of

adaptation and interpretation of difference-in-differences estimation, where exposure to treatment

is continuous and not binary?” In addition, Gardner (2022) discusses another dimension: what

adjustments should be made to DiD to apply it correctly in situations where policies are

implemented on various dates in different groups, and classic approaches are still not practical.

Research Designs and Evaluation Frameworks

Berhail and Katipo N. used the comparative observational design typical of

climatological study (Berhail & Katipo N., 2023). They retrieved long-term amounts of

precipitation and evapotranspiration. They used both the calculation of the short-term

precipitation index (SPI) and the long-term accumulation of evapotranspiration index (SPEI) to

make a side-by-side comparison of time and space over several decades. This theory-based plan

focused on internal validity, since the drought indices were habitually isolated and examined

their behaviour with different climatic parameters.

Instead, Callaway et al. (2024) suggested a methodological improvement of a DiD

applicable to continuous treatment through the generalization of the parallel trends assumption

and Gardner (2022) suggested a two-stage DiD to staggered treatment uptake across units. They

are not empirical applications of an intervention but instrumental designs aimed at enhancing

program evaluations, including the estimation of the policy impact when treatment is maximum-

based or implemented in stages. Program evaluations involving these designs aim to enhance

causal inference, but unlike the testing of instrumental concepts in isolation, they rely on

observational data.

5

Data Sources and Types

Metecorel Berhail and Katipoglu (2023) conducted a theoretical study based solely on

quantitative secondary data obtained from the meteorological station to calculate the SPI and

SPEI indexes at various time periods. They relied on climatic data from multiple decades in their

study, making the indices comparable between wet and dry years and across seasonal periods

within a single geographic area.

Conversely, the study is about statistical needs when looking at the data of policy

evaluations, typically quantitative, observational data, including the use or non-use of health

policies or the degree of environmental regulation. These methods are based on the assumption

that it is possible to have panel data with continuous treatment variables over time, and that there

are control variables to meet the assumption of parallel trends. These methods primarily employ

quantitative techniques. However, they may also be used in mixed-methods evaluations where

the design incorporates questionnaires, administrative data, or program logs as continuous

explanatory variables.

Data Collection and Processing Methods

The problem on which Berhail and Katipoğlu (2023) focused was conducted by using

one of the rigorous pipelines of data processing, in which monthly values of precipitation and

evapotranspiration were retrieved from the meteorological stations of the Wadi Mekerra basin.

They calculated SPI and SPEI at 1-month, 3-month, and 6-month intervals and compared their

effectiveness in detecting drought events, baselines, and intensity criteria. It involved descriptive

statistics, correlation tests, and extreme value analysis of which of the two indices was most

effective in fitting the observed results in terms of drought outcomes.

6

In the meantime, Callaway et al. (2024) and Gardner (2022) developed a theoretical

framework that did not involve any new data collection. They thus use simulated or historical

data to estimate the better DiD estimators. The concern of their methodological contributions is

directly linked to the way that researchers can organise their data collection procedure, that is, by

focusing on longitudinal measurement incorporation and being conscious of the treatment

schedules of the customers, consequently quantifying and comparing treatment between the units

and over time. This indirect effect on data collection ensures that evaluators can deliver studies

that isolate causal effects, even when treatment exposure fluctuates flexibly or sporadically

across groups.

Stakeholder Involvement and Implementation Roles

In theoretical studies, such as Berhail and Katipoglu (2023), there is little stakeholder

engagement. The major stakeholders include climatologists, government agencies seeking

improvements in drought assessment, and funding offices. Although the meteorological stations

provide raw data, they do not determine the design, interpretation, or conclusions of the study.

Consequently, studies continue to focus on the scientific validity of the indices rather than their

programmatic utility.

In comparison, program evaluation settings issue the question of many stakeholders,

i.e., policymakers, funders, as well as program managers and community members. The

innovations of both Callaway et al. (2024) and Gardner (2022) have obvious implications to the

stakeholders in the evaluation; though methodological studies instead of applied studies, they

offer superior instruments to policy analysts and decision-makers to interpret continuously

distributed policy parameters, such as the size of the subsidy or the amount of service provided.

Such evaluations usually include consultation of stakeholders in developing relevant thresholds

7

in treatments, agreements with agencies to facilitate access to data, and partners in interpreting

results. In this way, their proposed frameworks, even though they do not presuppose primary

program evaluations, serve to assist stakeholders in evaluation design and results dissemination.

Ethical and Legal Considerations in Research and Evaluation

The two types of research paradigms should align with ethical guidelines and legal

restrictions. Berhail and Katipoglu (2023) utilised publicly available climatic data, ensuring their

work is not associated with privacy issues, as the sources of data used and the computational

reproducibility of the results are appropriately cited. Nevertheless, they face the moral burden of

conducting a responsible reporting of results due to the overall import of drought evaluation in

the sensitive areas.

Conversely, in the case of continuous-treatment DiD policy analysis (as described by

Callaway et al., 2024, and Gardner, 2022), analysts primarily analyse sensitive administrative

data, such as tax records or health outcomes. In such instances, some of the key ethical

considerations include maintaining confidentiality, ensuring Institutional Review Board (IRB)

approval, and protecting against the disclosure of personally identifiable information. The

application of legal specifications like the Family Educational Rights and privacy Act (FERPA)

or the Health Insurance Portability and Accountability Act (HIPAA) also takes place. There is

complexity in consent when some participants are being treated continuously and others at

different levels, just as participants need to be well aware of how their data can be utilised.

Anonymous data transmission, data secrecy and disclosure, and data safety are keys to ethical as

well as legal conducts.

8

Conclusion

This detailed comparison highlights that theoretical studies, as demonstrated by Berhail

and Katipoglu (2023) in their climatological study, aim to enhance scientific learning in a

controlled and systematic environment. On the other hand, program evaluation methodological

developments, such as those of Callaway et al. (2024) and Gardner (2022), focus on enhancing

the causal inference instrument in the development of real-life policies. Theoretical research

provides precision regarding the concepts; these ideas are translated into practical methodologies

for applied contexts with the help of evaluation frameworks. Ethical rigour, appropriateness of

the methods, cooperation of the stakeholders, and legality are essential in both paradigms.

Research in the human services offers a combination of depth and effectiveness by integrating

theoretical insights into practice evaluation methods. Combinations of approaches contribute to

the increase in research utility and the effectiveness of programs, ultimately leading to changes in

the outcomes of served individuals and communities.

9

References

Berhail, S., & Katipoğlu, O. M. (2023). Comparison of the SPI and SPEI as drought assessment

tools in a semi-arid region: case of the Wadi Mekerra basin (northwest of

Algeria). Theoretical and Applied Climatology, 154(3-4), 1373-1393.

https://www.proquest.com/openview/8966a8bf574076f309539b82f62dcea0/1?cbl=48318

&pq-origsite=gscholar

Callaway, B., Goodman-Bacon, A., & Sant'Anna, P. H. (2024). Difference-in-differences with a

continuous treatment (No. w32117). National Bureau of Economic Research.

https://www.nber.org/papers/w32117

Gardner, J. (2022). Two-stage differences in differences. arXiv preprint arXiv:2207.05943.

https://doi.org/10.48550/arXiv.2207.05943

Hammer, M. R. (2023). The Intercultural Development Inventory: A new frontier in assessment

and development of intercultural competence. In Student learning abroad (pp. 115-136).

Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9781003447184-

7/intercultural-development-inventory-mitchell-hammer

Kalkbrenner, M. T. (2023). Alpha, omega, and H internal consistency reliability estimates:

Reviewing these options and when to use them. Counseling Outcome Research and

Evaluation, 14(1), 77-88. https://doi.org/10.1080/21501378.2021.1940118