A logic model is a tool that can be used in planning a program. Using a logic model, social workers can systematically analyze a propose

Excerpts from Measuring Program Outcomes: A Practical Approach © 1996 United Way of America

Introduction to Outcome Measurement

If yours is like most human service agencies or youth- and family-serving organizations, you regularly monitor and report on how much money you receive, how many staff and volunteers you have, and what they do in your programs. You know how many individuals participate in your programs, how many hours you spend serving them, and how many brochures or classes or counseling sessions you produce. In other words, you document program inputs, activities, and outputs. Inputs include resources dedicated to or consumed by the program. Examples are money, staff and staff time, volunteers and volunteer time, facilities, equipment, and supplies. For instance, inputs for a parent education class include the hours of staff time spent designing and delivering the program. Inputs also include constraints on the program, such as laws, regulations, and requirements for receipt of funding. Activities are what the program does with the inputs to fulfill its mission. Activities include the strategies, techniques, and types of treatment that comprise the program's service methodology. For instance, sheltering and feeding homeless families are program activities, as are training and counseling homeless adults to help them prepare for and find jobs. Outputs are the direct products of program activities and usually are measured in terms of the volume of work accomplished–for example, the numbers of classes taught, counseling sessions conducted, educational materials distributed, and participants served. Outputs have little inherent value in themselves. They are important because they are intended to lead to a desired benefit for participants or target populations. If given enough resources, managers can control output levels. In a parent education class, for example, the number of classes held and the number of parents served are outputs. With enough staff and supplies, the program could double its output of classes and participants. If yours is like most human service organizations, you do not consistently track what happens to participants after they receive your services. You cannot report, for example, that 55 percent of your participants used more appropriate approaches to conflict management after your youth development program conducted sessions on that skill, or that your public awareness program was followed by a 20 percent increase in the number of low-income parents getting their children immunized. In other words, you do not have much information on your program's outcomes. Outcomes are benefits or changes for individuals or populations during or after participating in program activities. They are influenced by a program's outputs. Outcomes may relate to behavior, skills, knowledge, attitudes, values, condition, or other attributes. They are what participants know, think, or can do; or how they behave; or what their condition is, that is different following the program. For example, in a program to counsel families on financial management, outputs–what the service produces–include the number of financial planning sessions and the number of families seen. The desired outcomes–the changes sought in participants' behavior or status–can include their developing and living within a budget, making monthly additions to a savings account, and having increased financial stability. In another example, outputs of a neighborhood clean-up campaign can be the number of organizing meetings held and the number of weekends dedicated to the clean-up effort. Outcomes–benefits to the target population–might include reduced exposure to safety hazards and increased feelings of neighborhood pride. The program outcome model depicts the relationship between inputs, activities, outputs, and outcomes. Note: Outcomes sometimes are confused with outcome indicators, specific items of data that are tracked to measure how well a program is achieving an outcome, and with outcome targets, which are objectives for a program's level of achievement. For example, in a youth development program that creates internship opportunities for high school youth, an outcome might be that participants develop expanded views of their career options. An indicator of how well the program is succeeding on this outcome could be the number and percent of participants who list more careers of interest to them at the end of the program than they did at the beginning of the program. A target might be that 40 percent of participants list at least two more careers after completing the program than they did when they started it.

Program Outcome Model

Resources dedicated to or consumed by the program money staff and staff time volunteers and volunteer time

facilities equipment and supplies

Constraints on the program laws regulations funders' requirements

What the program does with the inputs to fulfill its mission feed and shelter homeless families provide job training educate the public about signs of child abuse counsel pregnant women create mentoring relationships for youth

The direct products of program activities number of classes taught number of counseling sessions conducted number of educational materials distributed number of hours of service delivered number of participants served

Benefits for participants during and after program activities new knowledge increased skills changed attitudes or values

modified behavior

improved condition altered status

Why Measure Outcomes?

In growing numbers, service providers, governments, other funders, and the public are calling for clearer evidence that the resources they expend actually produce benefits for people. Consumers of services and volunteers who provide services want to know that programs to which they devote their time really make a difference. That is, they want better accountability for the use of resources. One clear and compelling answer to the question of "why measure outcomes?" is to see if programs really make a difference in the lives of people. Although improved accountability has been a major force behind the move to outcome measurement, there is an even more important reason: to help programs improve services. Outcome measurement provides a learning loop that feeds information back into programs on how well they are doing. It offers findings they can use to adapt, improve, and become more effective. This dividend doesn't take years to occur. It often starts appearing early in the process of setting up an outcome measurement system. Just the process of focusing on outcomes–on why the program is doing what it's doing and how participants will be better off–gives program managers and staff a clearer picture of the purpose of their efforts. That clarification alone frequently leads to more focused and productive service delivery. Down the road, being able to demonstrate that their efforts are making a difference for people pays important dividends for programs. It can, for example, help programs:

• Recruit and retain talented staff • Enlist and motivate able volunteers • Attract new participants • Engage collaborators • Garner support for innovative efforts • Win designation as a model or demonstration site • Retain or increase funding • Gain favorable public recognition

Results of outcome measurement show not only where services are being effective for participants, but also where outcomes are not as expected. Program managers can use outcome data to:

• Strengthen existing services • Target effective services for expansion • Identify staff and volunteer training needs • Develop and justify budgets • Prepare long-range plans • Focus board members' attention on programmatic issues

To increase its internal efficiency, a program needs to track its inputs and outputs. To assess compliance with service delivery standards, a program needs to monitor activities and outputs. But to improve its effectiveness in helping participants, to assure potential participants and funders that its programs produce results, and to show the general public that it produces benefits that merit support, an agency needs to measure its outcomes. These and other benefits of outcome measurement are not just theoretical. Scores of human service providers across the country attest to the difference it has made for their staff, their volunteers, their decision makers, their financial situation, their reputation, and, most important, for the public they serve.

Eight Steps to Success

Measuring Program Outcomes provides a step-by-step approach to developing a system for measuring program outcomes and using the results. The approach, based on methods implemented successfully by agencies across the country, is presented in eight steps, shown below. Although the illustration suggests that the steps are sequential, this is actually a dynamic process with a good deal of interplay among stages.

Example Outcomes and Outcome Indicators for Various Programs These are illustrative examples only. Programs need to identify their own outcomes and indicators, matched to and based on their own experiences and missions and the input of their staff, volunteers, participants, and others.

Type of Program Outcome Indicator(s)

Smoking cessation class

Participants stop smoking. • Number and percent of participants who report that they have quit smoking by the end of the course

• Number and percent of participants who have not relapsed six months after program completion

Information and referral program

Callers access services to which they are referred or about which they are given information.

• Number and percent of community agencies that report an increase in new participants who came to their agency as a result of a call to the information and referral hotline

• Number and percent of community agencies that indicate these referrals are appropriate

Tutorial program for 6th grade students

Students' academic performance improves.

• Number and percent of participants who earn better grades in the grading period following completion of the program than in the grading period immediately preceding enrollment in the program

English-as-a- second-language instruction

Participants become proficient in English.

• Number and percent of participants who demonstrate increase in ability to read, write, and speak English by the end of the course

Counseling for parents identified as at risk for child abuse or neglect

Risk factors decrease. No confirmed incidents of child abuse or neglect.

• Number and percent of participating families for whom Child Protective Service records report no confirmed child abuse or neglect during 12 months following program completion

Employee assistance program

Employees with drug and/or alcohol problems are rehabilitated and do not lose their jobs.

• Number and percent of program participants who are gainfully employed at same company 6 months after intake

Homemaking services

The home environment is healthy, clean, and safe. Participants stay in their own home and are not referred to a nursing home.

• Number and percent of participants whose home environment is rated clean and safe by a trained observer

• Number of local nursing homes who report that applications from younger and healthier citizens are declining (indicating that persons who in the past would have been referred to a nursing home now stay at home longer)

Prenatal care program

Pregnant women follow the advice of the nutritionist.

• Number and percent of women who take recommended vitamin supplements and consume recommended amounts of calcium

Shelter and counseling for runaway youth

Family is reunified whenever possible; otherwise, youths are in stable alternative housing.

• Number and percent of youth who return home • Number and percent of youth placed in alternative living arrangements who

are in that arrangement 6 months later unless they have been reunified or emancipated

Camping Children expand skills in areas of interest to them.

• Number and percent of campers that identify two or more skills they have learned at camp

Family planning for teen mothers

Teen mothers have no second pregnancies until they have completed high school and have the personal, family, and financial resources to support a second child.

• Number and percent of teen mothers who comply with family planning visits • Number and percent of teen mothers using a recommended form of birth

control • Number and percent of teen mothers who do not have repeat pregnancies

prior to graduation • Number and percent of teen mothers who, at the time of next pregnancy, are

high school graduates, are married, and do not need public assistance to provide for their children

Glossary of Selected Outcome Measurement Terms

Inputs are resources a program uses to achieve program objectives. Examples are staff, volunteers, facilities, equipment, curricula, and money. A program uses inputs to support activities. Activities are what a program does with its inputs-the services it provides-to fulfill its mission. Examples are sheltering homeless families, educating the public about signs of child abuse, and providing adult mentors for youth. Program activities result in outputs. Outputs are products of a program's activities, such as the number of meals provided, classes taught, brochures distributed, or participants served. A program's outputs should produce desired outcomes for the program's participants. Outcomes are benefits for participants during or after their involvement with a program. Outcomes may relate to knowledge, skills, attitudes, values, behavior, condition, or status. Examples of outcomes include greater knowledge of nutritional needs, improved reading skills, more effective responses to conflict, getting a job, and having greater financial stability. For a particular program, there can be various "levels" of outcomes, with initial outcomes leading to longer-term ones. For example, a youth in a mentoring program who receives one-to-one encouragement to improve academic performance may attend school more regularly, which can lead to getting better grades, which can lead to graduating. Outcome indicators are the specific items of information that track a program's success on outcomes. They describe observable, measurable characteristics or changes that represent achievement of an outcome. For example, a program whose desired outcome is that participants pursue a healthy lifestyle could define "healthy lifestyle" as not smoking; maintaining a recommended weight, blood pressure, and cholesterol level; getting at least two hours of exercise each week; and wearing seat belts consistently. The number and percent of program participants who demonstrate these behaviors then is an indicator of how well the program is doing with respect to the outcome. Outcome targets are numerical objectives for a program's level of achievement on its outcomes. After a program has had experience with measuring outcomes, it can use its findings to set targets for the number and percent of participants expected to achieve desired outcomes in the next reporting period. It also can set targets for the amount of change it expects participants to experience. Benchmarks are performance data that are used for comparative purposes. A program can use its own data as a baseline benchmark against which to compare future performance. It also can use data from another program as a benchmark. In the latter case, the other program often is chosen because it is exemplary and its data are used as a target to strive for, rather than as a baseline.

© 2014 Laureate Education, Inc. Page 1 of 1

© 2014 Laureate Education, Inc. Page 1 of 1

Figure 31.1

Logic Model

Logic Models

Karen A. Randolph

A logic model is a diagram of the relationship between a need that a

p rogram is designed to addret>s and the actions to be taken to address the need and achieve program outcomes. It provides a concise, one-page pic- ture of p rogram operations from beginning to end. The diagram is made up of a series of boxes that represent each of the program's com ponents,

inpu ts or resources, activities, outputs, and outcomes. The diagram shows how these components are connected or linked to one another for the purpose of achieving program goals. Figure 31.1 provides an example of the frame work for a basic logic model.

Th e program connections illustrate the logic of how program operations will result in client change (McLaughlin & Jordan, 1999). The connections show the "causal" relati on- ships between each of the program components and thus are referred to as a series of"if- then" sequence of changes leading to th e intended outco mes for the target client group (Chinman, hum, & Wandersman, 2004). The if-then statements represent a program's theory of change underlying an intervention. As such, logic models provide a framework that g uides the evaluation process by laying out important relationships that need to b e tested to demonstrate program results (Watso n, 2000).

Logic models come from the field of program evaluation. The idea emerged in response to the recognition among program evaluators regardin g the need to systema tize the p r ogram evaluation process (McLaughlin & Jordan, 20 04). Since then , logic models have become increasingly popular among program managers for program planning and to monitor program performance. With a growing emphasis on accountability and out- come measurement, logic models make explicit the entire change process, Lhe assu mp- tions t hat underlie this process, and the pathways to reach ing outcomes. Researchers have begun to use logic models for intervention research planning (e.g., Brown, Hawkins, Arthur, Brin ey, & Abbott, 2007).

The followin g sections provide a description of the components of a basic logic model and how these compon ents are linked together, its relationship to a p rogram's theory of

[ : Inputs 1–_.,•1 Ac~vities ,II—-.~•{ .Outputs ·11—~·1 Outcomes I AUTHOR'S NOTE: The author wishes to acknowledge Dr. Tony Tripodi for his though lful comments on a drafl of this chapter.

547

548 PART V • CONCEPTUAL RESEARCH

change, and its uses and benefits. The steps for creating a logic model as well as the chal- lenges of the logic modeling process will be presented. The chapter concludes with an example of how a logic model was u~cd to enhance program outcomes for a family liter- acy program.

Components of a Logic Model

Typically, a logic model has four components: inputs or resources, activities, outputs, and outcomes. Outcomes can be further classified into short-term outcomes, intermediate outcomes, and long-term outcomes based on the length of time it takes to reach these outcomes (McLa ughlin & Jordan , 2004) . The components make up the connection between the planned work and the intended results (W. K. Kellogg Foundation, 2004). The planned work includes the resources (the inp uts) needed to im plement the program as well as how the resources will be used (the activities) . The intended results include the outputs and outcomes that occur as a consequence of the planned work. Figure 31.2 expands on the model illuslrated in Figure 3 1.1 by adding examples of each component. This particular logic model, adopted from frec htling (2007), provides an illustration of the components of an intervention designed to prevent substance abuse and other prob- lem behaviors among a population of youth. The intervention is targeted toward improv- ing parenting skills, based on the assumption that positive parenting leads to prosocial behaviors among yo uth {Bahr, Hoffman, & Yang, 2005). The following section provides definitions and examples of each logic model component, using this illustration.

Resources Resources, sometimes referred to as inputs, in clude the human, fin ancial, organizational, and community asse ts that are available to a program to achieve its objectives (W. K. Kellogg Foundation, 2004). Resources are used to support and facilitate the program activities. They are usually categorized in terms of funding resou rces or in -kind contribu- tion s (Frechtling, 2007) .

Some resources, such as laws, regulations, and funding requirements, are external to the agency (United Way of America, 1996). Other resources, such as staff and money, are easier lo quantify than others (e.g., community awareness of the program; Mertinko, Novotney, Baker, & Lange, 2000). As Fn.:c:htli ng (2007) notes, it is important to clearly and tho roughly id ent ify the available resources during the logic modeling process because this information defines the scope and parameters of the program. Also, this inCormation is critical for others who may be interes ted in replicating the program. The logic model in Figure 31.2 includes fu nding as one of its resources.

Activities Activities represent a program's service methodology, showing how a program intends on using the resources described previously to carry out its work. Activities are also referred to as ac tion step!; (McLaughlin & Jordan, 2004). They are the highly specifi c tasks that p rogram staffs engage in on a daily basis to provide services to clients (Mertinko et al., 2000) . They include all aspects of pro gram implementation, the processes, tools, events, technology, and program actions. The ac tivities form the foundation toward facil- itating intended client changes or reaching oulcornes (W. K. Kellogg Fo undation, 2004). Some examples are establishing community councils, providing professional develop – ment training, or initiating a media campaign (Frechtling, 2007). Other examples are

CHAPTER 31 • l OCIC MO DELS 549

Inputs Activities Outputs Outcomes

Short Term Intermediate Long Term

Feedback Loop j _J

I Decreased

K~ Increased

I Develop and Numbe r of Increased

youth Funds .~ initiate ~edi a st~tions a~opti ng r– awareness f- positive 1—–+ of positive substance -~m~tg~– -.:::c -campatgn J pa renting parenti ng – abv?~d'

~-'.:-

/ I

Develop and Number of Increased distribute – 1> fact sheets 1- enrollment

fact sheets distributed in parenting programs

Fig ure 31.2 Example of l ogic Model With Com ponents, Two Types of Connections, and a Feedbaclc loop

providing shelter for homeless families, educating the public about signs of child abuse, or providing adult mentors for youlh {United Way of Ame rica, 1996) . Two activities, " Deve lop and initiate media campaign" and "Develop a

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.