Assignment: Respond to two of your colleagues posts in one or more of the following ways: ‘See attachment for colleagues respon
Assignment:
Respond to two of your colleagues’ posts in one or more of the following ways: "See attachment for colleagues responses"
· 3 – 4 paragraphs
· No plagiarism
· APA citing
Week 8 Discussion 1
Expanding Contexts for Evaluation
To prepare for this Discussion, pay particular attention to the following Learning Resources:
· Review this week’s Learning Resources, especially:
· Read Changing Role of Evaluation – See Word doc .
· Basic Principles of Management – See pdf
Assignment:
Respond to two of your colleagues’ posts in one or more of the following ways:
· Ask a clarifying question about your colleague’s description of their response
· Suggest one or more additional feedback your colleague could use regarding subject.
· Relate an example from your own experience of a positive, effective, and/or to what your colleague shared.
· 3 – 4 paragraphs
· No plagiarism
· APA citing
1st – Colleague – Stephen Jarman
Stephen Jarman
Top of Form
Hello everyone, here we are at the end of HRM652, Evaluating Results & Benefits…almost surreal it's gone by so fast! My reflections (and recollections) are very positive in terms of what I've learned. The assignments allowed me to direct our learning toward real-world applications.
In week one, I selected two evaluation tools that have already been developed within my workplace – the SWOT analysis tool, and annual cultural survey too. I analyzed each of these for their importance and benefits, how they are used for decision making, and the evaluation measurements that are used and others should be used.
In week two, I interviewed my son and small business owner as he has reached the point that other business owners often do as they realize their business might fail due to a lack of measurement and evaluation method. With the inputs from his statements I wrote a framework for him to implement an expanded and customized Measurement and Evaluation Success Case Methodology coupled with strategy development and deployment methodology, My son is using the paper to build on his early foundation of successes to be able to continue to thrive at any growth trajectory that he chooses.
On week three, the overview of Kirkpatrick’s Taxonomy Method in my paper was focused on each of the two Maine manufacturing locations that I am employed with by highlighting i) the reason that an HPT practitioner is advised to perform essential Needs and Impact Assessments for critical outcomes within both companies using Kirkpatrick’s Taxonomy Method, ii) described how the Needs and Impact Assessments align with both company’s values and strategies, and iii) outlined a plan for implementing a Needs and Impact Assessment using Kirkpatrick’s Taxonomy Method for both companies
The assignment for week four was a summarization of the six rules and techniques whereby I was able to pair each of them to a program that I am assigned as annual objectives at my workplace. I argued that applying as such to Performance Management is a requisite to Performance Improvement (that is a very distinction for an HPT practitioner to realize and address).
In week five, I used the assignment to review two key functional groups at my workplace that are used a robust ROI methodologies and argued for additional areas of the business that can benefit from an ROI approach.
The assignment in week six was not as helpful as the others. I took the opportunitiy to i) contrast between strategic and tactical side of using emerging technology used in the evaluation process in any industry, ii) describe the benefits and importance of evaluation within my workplace, iii) explain emerging technologies as strategic goals at my workplace, and iv) briefly describe differences in methods used for evaluation of a non-profit businesses versus a for-profit business. I did not find that I benefited from the work.
The week seven assignment was the most applicable and purposeful assignment of this course, and I ended up sending it to my director as a proposal as an outline for i) the justification for performing an Operational Excellence (OpEx) Assessment at my company's North American Distribution Center (DC), ii) provided a description of the OpEx Assessment framework, iii) outlined the recommended plan for a OpEx Assessment at the DC, iv) gave a recommendation for implementation of how to make subsequent performance improvements, and v) listed some challenges that the organization may experience as it follows through on an OpEx Assessment and performance implementation plan.
I do know that inputs and indicators selected and used must be appropriate and meaningful to the organization in order for the HPT practitioner to be able to provide insights and direct improvements at the right pace and sequences over time.
Thank you to everyone for enriching my educational journey in this course. I've learned from you as I planned in the first week by reading and considering your discussion posts each week. Hopefully we've all made a positive difference for each other.
Best wishes for our next courses, let's keep going and finish our masters!
Stephen
Bottom of Form
2nd – Colleague –
Susan Christmas
Week 8 Discussion
Top of Form
For the final week of our discussions, we are asked to do some reflecting on what we have learned throughout these last eight weeks. We are asked to expand upon how we might utilize our learning within the workplace. We are asked what we feel could be immediately applied and what could be applied in the future. We then must explain how our understanding of natural science and evaluation will personally benefit us as we build upon our career. Finally, we are asked to distinguish how the understanding of inputs and indicators in the evaluation process will help to improve ourselves, as well as our workplace, over time.
Utilize Learnings Within the Workplace
If I choose to return to the workforce, I feel that our lesson from Week 1is something I would definitely need to utilize within the workplace. As mentioned that week, there are three conditions that must be in place before an organization can say that performance is being managed: (1) goals that are clear and measurable; (2) measurement feedback must be provided to the performers so decisions can be made; and (3) if measurement feedback dictates a need for change then there must be an ability to control resources and conditions (Moseley & Dessinger, 2010).
Immediate and Future Application of Learnings
I would say most of our teachings throughout the last eight weeks can be immediately applied, at least to some degree. Many of the teachings are processes that take time to be fully implemented and monitored before it can be determined how beneficial that change may be.
Personal Benefit of Natural Science and Evaluation
I feel that my understanding of evaluation will personally benefit me as I build upon my career because I have a better understanding of how we should treat one another and how to act as a leader. Evaluations are part of improving the workplace and improving ourselves as human beings.
Improving Self and Workplace
Inputs and indicators should be designed to guide ourselves and the organization into the best direction overall. The inputs and indicators must be chosen based on information that has already been determined and how the organization can work towards improving performance.
References
Moseley, J. & Dessinger, J. (2010). Handbook of Improving Performance in the Workplace, Measurement and Evaluation (Volume 3) Hoboken: Wiley.
Bottom of Form
,
BASIC PRINCIPLES OF MONITORING
AND EVALUATION
Basic principles of monitoring and evaluation i
CONTENT
1. MONITORING AND EVALUATION: DEFINITIONS
2. THEORY OF CHANGE
3. PERFORMANCE MANAGEMENT SYSTEMS AND
PERFORMANCE MEASUREMENT
4. PERFORMANCE INDICATORS
4.1 Process (implementation) indicators
4.2 Process (implementation) indicators
4.3 Progression indicators (labour market attachment)
5. TARGETS, BASELINE AND DATA SOURCES
6. MEASURING RESULTS
Basic principles of monitoring and evaluation 1
1. MONITORING AND EVALUATION: DEFINITIONS
Youth employment programmes, like any other type of public
policy intervention, are designed to change the current situation of the target group and achieve specific results, like increasing employment or reducing unemployment. The key policy question is whether the planned results (outcomes) were actually achieved. Often, in fact, the attention of policy-makers and programme managers is focused on inputs (e.g. the human and financial resources used to deliver a programme) and outputs (e.g. number of participants), rather than on whether the programme is achieving its intended outcomes (e.g. participants employed or with the skills needed to get productive jobs).
Monitoring and evaluation are the processes that allow policy- makers and programme managers to assess: how an intervention evolves over time (monitoring); how effectively a programme was implemented and whether there are gaps between the planned and achieved results (evaluation); and whether the changes in well-being are due to the programme and to the programme alone (impact evaluation).
Monitoring is a continuous process of collecting and analysing information about a programme, and comparing actual against planned results in order to judge how well the intervention is being implemented. It uses the data generated by the programme itself (characteristics of individual participants, enrolment and attendance, end of programme situation of beneficiaries and costs of the programme) and it makes comparisons across individuals, types of programmes and geographical locations. The existence of a reliable monitoring system is essential for evaluation.
Evaluation is a process that systematically and objectively assesses all the elements of a programme (e.g. design, implementation and results achieved) to determine its overall worth or significance. The objective is to provide credible information for decision-makers to identify ways to achieve more of the desired results. Broadly speaking, there are two main types of evaluation: Performance evaluations focus on the quality of service delivery
and the outcomes (results) achieved by a programme. They typically cover short-term and medium-term outcomes (e.g. student achievement levels, or the number of welfare recipients who move into full-time work). They are carried out on the basis of information regularly collected through the programme monitoring system. Performance evaluation is broader than monitoring. It attempts to determine whether the progress achieved is the result of the intervention, or whether another explanation is responsible for the observed changes.
Impact evaluations look for changes in outcomes that can be directly attributed to the programme being evaluated. They estimate what would have occurred had beneficiaries not participated in the programme. The determination of causality between the programme and a specific outcome is the key feature that distinguishes impact evaluation from any other type of assessment.
2 Basic principles of monitoring and evaluation
Monitoring and evaluation usually include information on the cost of the programme being monitored or evaluated. This allows judging the benefits of a programme against its costs and identifying which intervention has the highest rate of return. Two tools are commonly used. A cost-benefit analysis estimates the total benefit of a
programme compared to its total costs. This type of analysis is normally used ex-ante, to decide among different programme options. The main difficulty is to assign a monetary value to “intangible” benefits. For example, the main benefit of a youth employment programme is the increase of employment and the earning opportunities for participants. These are tangible benefits to which a monetary value can be assigned. However, having a job also increase people’s self-esteem, which is more difficult to express in monetary terms as it has different values for different persons.
A cost-effectiveness analysis compares the costs of two or more programmes in yielding the same outcome. Take for example a wage subsidy and a public work programme. Each has the objective to place young people into jobs, but the wage subsidy does so at the cost of $500 per individual employed, while the second costs $800. In cost-effectiveness terms, the wage subsidy performs better than the public work scheme.
Basic principles of monitoring and evaluation 3
2. THEORY OF CHANGE
A theory of change describes how an intervention will deliver the planned results. A causal/result chain (or logical framework) outlines how the sequence of inputs, activities and outputs of a programme will attain specific outcomes (objectives). This in turn will contribute to the achievement of the overall aim. A causal chain maps: (i) inputs (financial, human and other resources); (ii) activities (actions or work performed to translate inputs into outputs); (iii) outputs (goods produced and services delivered); (iv) outcomes (use of outputs by the target groups); and (v) aim (or final, long-term outcome of the intervention).
In the result chain above, the monitoring system would continuously track: (i) the resources invested in/used by the programme; (ii) the implementation of activities in the planned timeframe; and (iii) the delivery of goods and services. A performance evaluation would, at a specific point of time, judge the inputs-outputs relationship and the immediate outcomes. An impact evaluation would provide evidence on whether the changes observed were caused by the intervention and by this alone.
FINAL OUTCOMES
OUTCOMES
EVALUATION MONITORING
ACTIVITIES INPUTS OUTPUTS
Final programme
goals, typically achieved in the
long-term
Results likely to be achieved
when beneficiaries
use outputs
Tangible goods or services the
programme produces or
delivers
Action taken/work
performed to transform inputs into
outputs
Available resources, including
budget and staff
Implementation Results
Figure 1. Results chain
Basic principles of monitoring and evaluation 4
3. PERFORMANCE MANAGEMENT SYSTEMS AND PERFORMANCE MEASUREMENT
Performance management (or results-based management) is a
strategy designed to achieve changes in the way organizations operate, with improving performance (better results) at the core of the system. Performance measurement (performance monitoring) is concerned more narrowly with the production of information on performance. It focuses on defining objectives, developing indicators, and collecting and analysing data on results. Results-based management systems typically comprise seven stages:
1. Formulating objectives: identifying in clear, measurable terms the results being sought and developing a conceptual framework for how the results will be achieved.
2. Identifying indicators: for each objective, specifying exactly what is to be measured along a scale or dimension.
3. Setting targets: for each indicator, specifying the expected level of results to be achieved by specific dates, which will be used to judge performance.
4. Monitoring results: developing performance-monitoring systems that regularly collect data on the results achieved.
5. Reviewing and reporting results: comparing actual results against the targets (or other criteria for judging performance).
6. Integrating evaluations: conducting evaluations to gather information not available through performance monitoring systems.
7. Using performance information: using information from monitoring and evaluation for organizational learning, decision- making and accountability.
The setting up a performance monitoring system for youth employment programmes, therefore, requires: clarifying programme objectives; identifying performance indicators; setting the baseline and targets, monitoring results, and reporting.
In many instances, the objectives of a youth employment programme are implied rather than expressly stated. In such cases, the first task of performance monitoring is to articulate what the programme intends to achieve in measurable terms. Without clear objectives, in fact, it becomes difficult to choose the most appropriate measures (indicators) and express the programme targets.
S tr
a te
g ic
p la
n n
in g
P e
rf o
rm a
n c
e m
e a
s u
re m
e n
t
R E
S U
L T
S -B
A S
E D
M A
N A
G E
M E
N T
Figure 2 Steps of performance management systems
5 Basic principles of monitoring and evaluation
4. PERFORMANCE INDICATORS
Performance indicators are concise quantitative and qualitative measures of programme performance that can be easily tracked on a regular basis. Quantitative indicators measure changes in a specific value (number, mean or median) and a percentage. Qualitative indicators provide insights into changes in attitudes, beliefs, motives and behaviours of individuals. Although important, information on these indicators is more time-consuming to collect, measure and analyse, especially in the early stages of programme implementation.
Box .1. Tips for the development of indicators
Relevance. Indicators should be relevant to the needs of the user and to the purpose of monitoring. They should be able to clearly indicate to the user whether progress is being made (or not) in addressing the problems identified.
Disaggregation. Data should be disaggregated according to what is to be measured. For example, for individuals the basic disaggregation is by sex, age group, level of education and other personal characteristics useful to understanding how the programme functions. For services and/or programmes the disaggregation is normally done by type of service/programme.
Comprehensibility. Indicators should be easy to use and understand and data for their calculation relatively simple to collect.
Clarity of definition. A vaguely defined indicator will be open to several interpretations, and may be measured in different ways at different times and places. It is useful in this regard to include the source of data to be used and calculation examples/methods. For example, the indicator “employment of participants at follow-up” will require: (i) specification of what constitutes employment (work for at least one hour for pay, profit or in kind in the 10 days prior to the measurement); (ii) a definition of participants (e.g. those who attended at least 50 per cent of the programme); and (iii) a follow-up timeframe (six months after the completion of the programme). Care must also be taken in defining the standard or benchmark of comparison. For example, in examining the status of young people, what constitutes the norm – the situation of youth in a particular region or at national level?
The number chosen should be small. There are no hard and fast rules to determine the appropriate number of indicators. However, a rule of thumb is that users should avoid two temptations: information overload and over-aggregation (i.e. too much data and designing a composite index based on aggregation and weighting schemes which may conceal important information). A common mistake is to over-engineer a monitoring system (e.g. the collection of data for hundreds of indicators, most of which are not used). In the field of employment programmes, senior officials tend to make use of high-level strategic indicators such as outputs and outcomes. Line managers and their staff, conversely, focus on operational indicators that target processes and services.
Specificity. The selection of indicators should reflect those problems that the youth employment programme intends to address. For example, a programme aimed at providing work experience to early school leavers needs to incorporate indicators on coverage (how many among all school leavers participate in the programme), type of enterprises where the work experience takes place and the occupation, and number of beneficiaries that obtain a job afterwards by individual characteristics (e.g. sex, educational attainment, household status and so on).
Cost. There is a trade off between indicators and the cost of collecting data for their measurement. If the collection of data becomes too expensive and time consuming, the indicator may ultimately lose its relevance.
Technical soundness. Data should be reliable. The user should be informed about how the indicators were constructed and the sources used. A short discussion should be provided about their meaning, interpretation, and, most importantly, their limitations. Indicators must be available on a timely basis, especially if they are to provide feedback during programme implementation.
Forward-looking. A well-designed system of indicators must not be restricted to conveying information about current concerns. Indicators must also measure trends over time.
Adaptability. Indicators should be readily adaptable to use in different regions and circumstances.
Source: adapted from Canadian International Development Agency (CIDA), 1997. Guide to Gender-Sensitive Indicators (Ottawa, CIDA).
Basic principles of monitoring and evaluation 6
When choosing performance indicators, it is important to identify indicators at all levels of the results chain, and not just at the level of outcomes. Information on process is useful for documenting programme implementation over time and explaining differences across programme sites. Information on individual participants (e.g. sex, age group, national origin, medical condition, educational attainment, length of unemployment spells and so on) allows users to judge compliance with targeting criteria. Some examples of the most common implementation indicators are shown in the Table 1 below.
Process indicators Calculation method Disaggregation
1 Composition of entrants,
participants, completers *
Number of entrants in period t*100
= —————————
Total number of entrants in period t
by type of programme
by characteristics of individuals
Programme: training, subsidy, self-
employment, etc.
Individuals by sex, age group, education
level, unemployment duration, type of
disadvantage, prior occupation/work
experience
2 Stock variation of entrants,
participants, completers
Number of entrants in period t
= —————————
Number of entrants in period t-1
As above
3 Inflow of entrants (or
participants)
Number of new entrants in period t
= —————————
Stock of entrants end of period t-1
As above
4 Degree of coverage of target
population (entrants,
participants, completers)
Number of programme entrants*100
= —————————
Total targeted population
As above
5 Implementation Number of implemented actions
= ————————–
Number of planned actions
As above
6 Average cost per entrant,
participant, completer
Total cost of programme
= —————————
Total number of entrants
By programme
Note: * Entrants are all individuals who start a specific programme. Participants are all individuals who entered and attended the programme for a minimum period of time (usually determined by the rules of the programme as the minimum period required to produce changes, for example 50 per cent of the programme duration). Completers are those who completed the whole programme. Dropouts, usually, are those who left the programme before a minimum period of attendance established by the rules of the programme (e.g. the difference between entrants and participants).
Table.1. Example of common process (implementation) indicators (measurement and disaggregation)
4.1 PROCESS (IMPLEMENTATION)
INDICATORS
7 Basic principles of monitoring and evaluation
The indicator in the first row, for example, serves to determine whether the targeting rules of the programme are being complied with. For instance, in a youth employment programme targeting individuals with less than primary education, the share of entrants by this level of education over the total will determine if eligibility rules are being followed and allow tracking of sites with the best/worst compliance.
The indicators in the second and third rows serve to measure the evolution of the programme’s intake. It is normal, in fact, to see increases in intake as the programme matures. The time t may be any time interval (yearly, quarterly or monthly). The indicator in the fourth row serves to measure the overall coverage of the programme. Depending on its scope, the denominator can be the total number of youth (in a country, region, province or town) or only those who have certain characteristics (e.g. only those who are unemployed, workers in the informal economy, individuals with a low level of education). The indicator in the fifth row serves to measure the pace of implementation compared to the initial plan, while the indicator in the last row is used to calculate overall costs.
Since the overarching objective of youth employment programmes is to help young people get a job, the most significant outcome indicators are: (i) the gross placement (employment) rate by individual characteristics and type of programme; (ii) average cost per young person placed; and (iii) the level of earnings of youth participants employed. The more disaggregated the data, the better, as this allows comparison across individuals, programmes and geographical locations.
Calculation methods and disaggregation are shown in Table 2 below.
Outcome
indicators
Calculation
method
Disaggregation
1 Gross placement rates
(individuals) Number of
placements*100
= ———————–
Total number
participants
(including dropouts)
by type of programme
by characteristics of individuals
by type of job
Programme: training, subsidy, self-employment,
public work scheme
Individuals by sex, age group, education level,
unemployment duration, type of disadvantage,
prior occupation/work experience
Jobs by economic sector and size of the
enterprise, occupation, contract type and
contract duration
Table 3: Outcome indicators (measurement and disaggregation)
4.2 PROCESS (IMPLEMENTATION) INDICATORS
Basic principles of monitoring and evaluation 8
The above-mentioned disaggregation also allows data users to judge the “quality” of the results achieved. The use of total placement as an indicator of performance, in fact, has two main shortcomings. The first is the likely prevalence of short-term employment and the likelihood that beneficiaries re-enter unemployment soon after the end of the programme. The second is the lack of distinction between “easy- to-place” youth (who would eventually get a job also without the programme) and “disadvantaged” youth (who are likely to experience long spells of unemployment if they are not helped). The first issue results in “gaming” behaviour, for example, administrators may be tempted to “cheat” the system by focusing on short-term placement (with no attention to quality) to achieve programme targets. The second gives rise to “creaming” (or cream-skimming), namely the selection for programme participation of those youth most likely to succeed, as compared to those who most need the programme.
The disaggregation proposed in Table 2 corrects these shortcomings by requiring collection of information on the characteristics of individuals employed and the type of jobs they perform. Calculation of hourly wages helps to measure the welfare gains more accurately than total earnings, as young workers may have higher earnings only because they work longer hours.
Cost is another important measure: it allows users to decide whether a programme is cost-effective (e.g. whether the rate of return in terms of placement justifies the resources invested).1 Usually, the overall costs of a programme are compared to those of other programmes with similar objectives and target groups. Overall costs include: 1) the disbursements made to service providers (e.g. the payment made to a training centre to conduct a vocational training course) or to other agencies (e.g. the cost of insuring participants during programme participation); 2) payments made to individual participants (e.g. the reimbursement of transport costs incurred to reach the site of training, subsidies for living costs and so on); and 3) the administrative cost of running the programme.
2 Earnings Number of individuals
placed in a job and earning
(hourly) wages over the
minimum*100
= ———————–
Number of placements
by type of programme
by characteristics of individuals
by type of jobs
3 Cost per placement Total cost
= ———————–
Number of placements
by type of programme
1 In terms o
Collepals.com Plagiarism Free Papers
Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.
Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS
Why Hire Collepals.com writers to do your paper?
Quality- We are experienced and have access to ample research materials.
We write plagiarism Free Content
Confidential- We never share or sell your personal information to third parties.
Support-Chat with us today! We are always waiting to answer all your questions.
