Anticrime/Prevention Program: Part 2 ? Evaluation Strategy & Logic Model Assignment Instructions DUE DATE: by 10am FRIDAY January 26, 2023. NO LATE WORK!!! Overview This assignment requir

15

PLANNING A PROGRAM EVALUATION

2 Keywords problem-oriented policing (POP) SARA S.M.A.R.T. logic model

CHAPTER OUTLINE

Introduction 15

Problem-Oriented Policing 16

Planning an Evaluation Strategy 18

Goal Setting: S.M.A.R.T. 18

Goal–Objective Relationship 19

Develop Evaluation Measures 19

Data Collection 20

Determine Analysis Methods 21

Logic Model 22

Politics of Evaluation Research 24

Ethical Issues in Evaluation Research 25

Ethics of Conducting Research 25

Ethics and Social Relationships in Evaluation Research 28

Summary 29

Discussion Questions 29

References 30

Introduction Criminal justice programs are designed to address a particular need

and solve a defi ned problem. Crime problems are often intractable and diffi cult to address. Strategic thinking, planning, and operations are

Planning proceeds step by step and each step must be evaluated before the next step can be taken.

—Edward Suchman (1987)

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

16 Chapter 2 PLANNING A PROGRAM EVALUATION

required to address the sources of crime problems, both individual and systemic. Panaceas and “silver-bullet” solutions to crime problems are often sought but never found. Collection of data and the compilation of research evidence typically are a part of program evaluation.

Th is is where program planning begins and program evaluation fol- lows. Th is process has been adequately summarized under the prob- lem-oriented policing (POP) acronym (SARA) ( Goldstein, 1990 ): ● Scanning: Identify recurring problems and how they aff ect com-

munity safety. ● Analysis: Determine the causes of the problem. ● Response: Seeking out, selecting, and implementing activities to

solve the problem. ● Assessment: Determine if the response was eff ective or identify

new strategies. Th e aim of this process is to collect data that are related to the prob-

lem, determine the validity of these data, trace causal relationships that could lead to problem identifi cation and program development to address the problem, and then determine if the program was eff ective in solving the problem.

Scanning involves looking for and identifying problems. Th e ini- tial analysis is made to determine if a problem exists and whether a detailed analysis is required to uncover it. Problems are also prioritized and personnel allocated.

Analysis is concerned with learning about the causes, scope, and eff ects of the problem. Who are the actors involved in the problem (both off enders and victims)? What are the specifi c incidents of the problem and what is the sequence of events leading to it? What responses have been made by the community and government agencies?

Response involves acting to alleviate the problem. Planned solu- tions can be organized regarding total elimination of the problem, material reduction of the problem, reduction of the harm caused by the problem, using the best possible solution to the problem, and even removing the problem from police consideration.

Assessment is where program evaluation comes in. Th at is, did the response work?

Problem-Oriented Policing POP requires that the police develop a systematic process for exam-

ining and addressing the problems that the public expects them to handle. It requires identifying these problems in more precise terms, researching each problem, documenting the nature of the current police response, assessing its adequacy and the adequacy of existing authority and resources, engaging in a broad exploration of alternatives

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

Chapter 2 PLANNING A PROGRAM EVALUATION 17

to present responses, weighing the merits of these alternatives, and choosing from among them ( Goldstein, 1979 ).

As defi ned by the San Diego, CA, police department ( Capowich & Roehl, 1994, pp. 127–128 ):

POP emphasizes identifying and analyzing problems (criminal, civil, or public nuisance) and implementing solutions to resolve the underlying causes of the problem. It emphasizes proactive intervention rather than reactive responses to calls for service, resolution of root causes rather than symptoms, and use of multiparty, community-based problem solving rather than a unilateral police response. POP focuses on a problem in a long-term, comprehensive manner, rather than handling the problem as a series of separate incidents to be resolved via arrest or other police action.

In San Diego, the police focused on street robberies for one year. Research determined that calls for service did not decrease over the one-year period. Yet, the indicators revealed that the POP approach improved the circumstances for those who used the stations and reduced the police workload at those stations. However, the role played by citizen groups in this project was very limited.

Elsewhere, research on POP reports positive results. In both Newport News, VA, and Baltimore County, MD, offi cers concentrated on under- lying causes of crime. Th ey collected information and enlisted the sup- port of public and private agencies. As a result, both crime and fear of crime were substantially reduced in both areas ( Eck & Spelman, 1987 ). Th e Center for Problem-Oriented Policing provides copies of evaluations ( www.popcenter.org/casestudies/ ) of POP operations that have eff ec- tively addressed such disparate problems as drunken driving, repeat sex off enders, construction site thefts, thefts from cars in parking facilities, burglary of single-family houses, drug dealing in apartment complexes, loud car stereos, street prostitution, and residential speeding.

Basically, there are two ways to defi ne a policing problem. In clear terms, a policing problem exists when something is going wrong for someone somewhere and somebody wants the police to do something about it. Problems can be perceived as a group of events that are simi- lar in one or more ways that are harmful to members of the public and that citizens expect the police to handle.

Th e problem analysis process must ferret out relationships among the variables collected in the data to determine the extent and nature of the problem. It is also important to ascertain if the problem can be adequately measured and thus serve as the basis for an outcome evalu- ation. First, there are concerns regarding the data: Can adequate data be obtained? Second, there are resource questions to be addressed: Does the police have the tools necessary to collect data and analyze it? What amount of time and money are necessary to conduct the study? Hopefully, the problem addressed is signifi cant enough to advance

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

18 Chapter 2 PLANNING A PROGRAM EVALUATION

knowledge and practice in criminal justice and the analysis will con- tribute to future attempts to address the problem in question. In this manner, evaluation research not only examines the eff ectiveness of a particular program or policy, but also adds to crime prevention knowl- edge by revealing those strategies that have proven to be able to solve a crime problem.

Planning an Evaluation Strategy Th e program evaluator must fi rst develop a plan to conduct the

research process. Basically, evaluation planning involves the comple- tion of fi ve basic steps: 1. State the goals of the program in clear and measurable (quantifi able)

terms. 2. Determine the relationship between goals and objectives. 3. Develop evaluation measures. 4. Determine the data to be collected on these measures. 5. Determine analysis methods.

Th ese steps help clarify the aims and abilities of the program in question to address the crime problem.

Goal Setting: S.M.A.R.T. Program goals are not often stated in clear terms, and they often

express the wishes of a particular group rather than a defi nite target. Th ey are not stated in a manner that will make it possible to deter- mine whether the program is eff ective. Here, the evaluator attempts to state the goals of the program in measurable terms that indicate their achievement. Th ey are often stated as a percentage to be attained over a specifi ed time period. Th ey should be attainable, reasonable, and refl ect what can be achieved if the program is eff ective. For example, the goal of a burglary prevention program could be stated in measurable terms as: To reduce the number of reported burglaries in the target area by 25% over a one-year period. Note that this statement indicates what the unit of measurement is (the number of reported burglaries) as well as the percentage target (25%) and the time period (one year after the pro- gram begins). However, what is the basis for such an expectation? Has previous research on burglary prevention programs revealed that a 25% reduction is possible? If the target specifi ed by the goal is unrealistic and unattainable, its use as a performance benchmark is worthless.

One method to avoid these problems in goal setting is to follow the acronym S.M.A.R.T, as shown in Figure 2.1 . (see Locke & Latham, 2013 ). Well-defi ned goals should be: ● Specifi c: Goals must be clear and specifi c. Th ey should represent

what the program is trying to accomplish and accurately measure successful operations.

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

Chapter 2 PLANNING A PROGRAM EVALUATION 19

● Measureable: If the goal is not measurable, there is no way to gauge progress.

● Attainable: Again, goals must be realistic. Some eff ort and stretch- ing is always in order, but goals should neither be too high or too low so they become meaningless.

● Relevant: Th e goals should represent and mirror the vision and mission of the program.

● Time-bound: Goals must have starting and end points and the duration of the measurement should be stated clearly. Formulating goals in this fashion will ensure good performance

by letting stakeholders know what is expected of them. Of course, they will also guide the course of the impact evaluation.

Goal–Objective Relationship In terms of our example, the burglary prevention program is

designed to reduce the incidence of this type of crime in the geographic area served by the program. Th is analysis can be extended to com- pare the incidence of burglary citywide in the areas not served by the program.

Develop Evaluation Measures Th e next step is to identify the evaluation measures for the pro-

gram under consideration – in this instance, a burglary prevention program. Here, the measures can take three basic forms: ● Eff ectiveness: Th ese measures determine the degree of success

of the program in dealing with the problem at hand. Here, we have stated that the quantifi able goal of the burglary prevention program is to reduce the incidence of reported burglaries in the targeted area by 25% over a one-year period.

Sp

ec ific

T im

e -bound

Relevant Atta

in a

bl e

Measurable

Smart goals

Figure 2.1 S.M.A.R.T. goals.

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

20 Chapter 2 PLANNING A PROGRAM EVALUATION

● Effi ciency: Th ese measures should indicate how well the program has been implemented and whether it has been implemented according to the original plan for the program.

● Attitudinal: Th ese measures can indicate whether the program has been successful by assessing the attitudes of the program’s clients. In our example, one way to accomplish this would be to conduct a before-and-after survey of the fear of crime among residents of the targeted area to determine if they felt safer after the burglary pre- vention program was implemented. In addition, the evaluator should consider the impact of factors that

occurred during the implementation of the program that could also aff ect these measures. For example, if the state in which the burglary prevention program was implemented passed a “rob a house, go to prison” (i.e., mandatory incarceration) bill, it may have an impact on the incidence of the number of reported burglaries.

Valid performance measures should have the following character- istics. First, they should be credible—accurate and relevant represen- tations of both the quality and quantity of services provided by the program. Second, they should provide a fair indication of program performance and refl ect the factors and operations that program administrators can truly infl uence and control. Th ird, they should be clear—easy to utilize and comprehend and practical to administer and implement.

Data Collection Th e fourth step in this process is to determine the data necessary to

perform the evaluation, the constraints on their availability, how these data will be collected and managed, and a method to determine the validity of these measures. In terms of understanding the problem that the proposed program addresses, four strategies are recommended for the evaluator ( Bickman, Rog, & Hedrick, 1998, p. 7 ): ● Hold discussions with research clients or sponsors to obtain the

clearest possible picture of their concerns. ● Review the relevant literature on the subject. ● Gather current information from experts and major interested

parties on the issue. ● Conduct information-gathering visits and observations to obtain

a real-world sense of the context, and talk with persons actively involved in the issue. In our example, it will be necessary to determine the baseline level

of reported burglaries in the target area prior to program implementa- tion and then the same measure after the program has been in place for one year. It will be necessary for the evaluator to establish methods to obtain these measures from the police department in question and

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

Chapter 2 PLANNING A PROGRAM EVALUATION 21

to determine how they collect and report information on reported bur- glaries in their jurisdiction. Th e evaluator must determine if these mea- sures are valid indicators of the incidence of burglaries in the area or whether it will be necessary to conduct a victimization survey of homes in the target area. Such a survey could determine whether burglaries occurred that were not reported to the police as well as the previously mentioned assessment of the fear of crime in the area (both before and after the program was implemented).

Th e cost of obtaining these data must also be considered by the evaluator. Obtaining the data from the police is much less costly than the possible victimization survey of homes in the target area. Burglary data would be regularly collected by the police, but the survey must be conducted by the evaluator. In any event, the evaluator must design an information system to collect all forms of data required by the program evaluation in a format that is amenable to statistical analysis and com- puterized. Th e availability of computers (tablet, laptop, or desktop) and statistical analysis programs (Excel, SPSS, SAS) helps to simplify this task.

Data must be carefully collected with an eye toward quality control. Procedures to collect the data must be established and variables must be explicitly defi ned in measurable terms so that the meaning of the data is clear and will be followed carefully. Th e source of the data must be clearly identifi ed, and the process of data storage, maintenance, and processing must be specifi ed.

Determine Analysis Methods Th e fi nal step is the determination of quantitative and/or qualita-

tive methods of analysis that will be utilized in the evaluation. Th ese methods will be determined in part by the evaluation design, the type of evaluation measures used, and their validity and reliability. Th e eval- uator should consider how the evaluation measures will be calculated and if these measures can be combined.

In our burglary prevention program example, a before-and-after comparison of the number and rate of reported burglaries before and one year after program implementation in terms of the expressed target of a 25% reduction is a rather simple comparison. However, other points of comparison could be added by conducting the same method of analysis for the areas of the city not served by the pro- gram. In addition, the evaluator could conduct a regression analysis based on past levels of reported burglaries to determine whether the number of reported burglaries after the one-year period of imple- mentation compared to the expected number of reported burglar- ies calculated by the regression analysis. In addition, the proposed victimization survey could provide another level of comparison by asking the respondents in the area their experiences with burglary

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

22 Chapter 2 PLANNING A PROGRAM EVALUATION

victimization over the time period in question, as well as their fear of crime over this period.

One of the mechanisms used to display and describe the under- lying theory and purpose of a program prior to conducting program evaluation is the use of a logic model.

Logic Model Th e process of establishing goals, objectives, and performance

indicators requires that the theory that provides the basis for the program must be specifi ed. In turn, the theory informs program operations by: ● Driving the selection of treatments. ● Clarifying the description of the services provided to clients with

defi ned needs. ● Helping to determine what variables need to be measured. ● Driving how one interprets a simple comparison of the outcomes

of two programs to deeper analyses in terms of research on the topical area in general. Th ese are also the elements of the logic model of a program—the

framework that shows how the program might theoretically produce the desired outcomes and impacts ( Boruch, 1998, p. 172 ).

Th e logic model specifi es the conceptual framework of an evalua- tion by establishing the variables to be measured and the expected rela- tionships among them. Th e logic model provides an explanation of how the program is expected to work and how the program goals, processes, resources, and outcomes will provide the direction for the program. Th us, they provide a clear “roadmap” of what is planned and expected results—a review of the strength of the connection between activi- ties and outcomes ( Knowlton & Phillips, 2013, p. 5 ). As a diagram, the logic model demonstrates the cause-and-eff ect mechanism between how the program will meet certain needs to achieve desired outcomes ( Davidson, 2005, p. 38 ). It focuses the evaluation of the program’s eff ec- tiveness and can communicate changes in its implementation over the life of the program ( Bickman et al., 1998, p. 8 ).

Table 2.1 presents a logic model for adult drug court programs as proposed by the National Institute of Justice. First is the program inputs, the resources required to operate the program. Th en, the pro- gram activities are the components of the program. Next, program outputs indicate the work done and the results of activities. Finally, related to outcomes are the short-term and long-term outcomes that represent the objectives of the program. Note that both the short-term and long-term program outcomes are measures of program perfor- mance for which data must be collected and compiled on program operations and activities. Th us, they will serve as the basis for the assessment of program performance and must be collected in a timely

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

Chapter 2 PLANNING A PROGRAM EVALUATION 23

fashion—hopefully in a computer-based information system operated and maintained by the adult drug court program. Th e external factors listed in Table 2 refer to outside forces that can aff ect the drug court program. For example, both the community and the courthouse work group could oppose or support the creation of a drug court. In addition, the legal/penal code of the state must be consulted to determine if the operations of the drug court are permissable.

By specifying the relationships among these key factors, the logic model clarifi es why the program was adopted and how it is expected to work. Th us, it serves as a blueprint for the evaluation of the eff ec- tiveness of the program.

Table 2.1 Adult Drug Court Program Logic Model

Inputs Activities Outputs

Short-Term

Outcomes

Long-Term

Outcomes

External

Factors

Probation Risk/needs

assessments

Program intake

screen

Recidivism

in-program

Recidivism

post-program

Community

Community Judicial

interaction

Program

admission

Alcohol and

other drug use

in-program

Alcohol and

other drug use

post-program

Legal/penal

code

Public

resources

Alcohol and other

drug monitoring

(including

testing)

Court

appearances

Supervision

violation

Program/

graduation

termination

Courthouse

Courthouse Community

supervision

Treatment

admission

Program violation Probation

revocation/

successful

termination

Treatment Graduated

sanctions or

incentives

(including jail)

Alcohol and other

drug tests

Treatment

retention

Jail/prison

imposed

Jail Alcohol and other

drug treatment

services

Probation

contracts

Skills

development

Employment,

education,

housing, and

health

Grant funds Ancillary services Classes attended Service needs met

Technical

assistance

Services accessed Criminal thinking

Jail stays

Source: National Institute of Justice, http://www.nij.gov/topics/courts/drug-courts/full-logic-model.htm .

Vito, G. F., & Higgins, G. E. (2014). Practical program evaluation for criminal justice. Taylor & Francis Group. Created from liberty on 2023-01-26 20:17:10.

C op

yr ig

ht ©

2 01

4. T

ay lo

r &

F ra

nc is

G ro

up . A

ll rig

ht s

re se

rv ed

.

24 Chapter 2 PLANNING A PROGRAM EVALUATION

Politics of Evaluation Research Up to this point, we have presented a rational model of program

evaluation. But it must be stressed that programs are political ani- mals. Th e reputations and egos of program sponsors and adminis- trators are tied to the success of the proposed program. It is unlikely that these groups will approach the evaluation in an experiment

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.