After watching the video Incorporating Supervision Best Practices, create a competency checklist that would work in your setting to assess the performance of your staff. Include at least 10

bs_bs_banner

Australian Psychologist (2015) © 2015 The Australian Psychological Society

1

ORIGINAL ARTICLE

Goal-setting, Feedback, and Assessment Practices Reported by Australian Clinical Supervisors Craig J Gonsalvez,1 Talia Wahnon,2 and Frank P Deane2

1School of Psychology, University of Western Sydney, and 2School of Psychology, University of Wollongong

Objective: The objective of the current study was to compare current supervisory practices in Australia against those derived from pedagogic principles and/or practice guidelines recommended by experts in the field. Three core supervisor competencies, namely goal-setting, providing formative feedback, and conducting summative assessments, were chosen for scrutiny. Methods: One hundred thirteen accredited psychology supervisors in Australia completed an online questionnaire that had supervisors report details about their goal-setting, feedback, and summative assessment practices in supervision. Results: Several aspects pertaining to summative assessment require improvement. Specifically, the limited use of observational techniques was inconsistent with principles of competency-based pedagogies and with recommendations by experts. A significant percentage of supervi- sors believed that summative ratings of trainee competence conducted by themselves (58%) and by their peers (66%) were compromised by leniency effects. Further, half the supervisors surveyed reported that summative assessments were made difficult by little or no guidance from training institutions about the benchmarks trainees were expected to meet at the end of placements. Conclusions: Supervisory practices concerning goal-setting were generally consistent with best-practice guidelines derived from the literature. However, improvements are warranted in key supervisory practices, including more frequent use of observational techniques to inform formative feedback, more effective strategies to counter leniency in summative assessment, and better communication between training institutions and supervisors.

Key words: clinical supervision; formative feedback; placement evaluation; practitioner training; summative assessment; supervisor training.

What is already known on this topic

1 The need to better understand current supervision practices is paramount particularly given the requirements for supervi- sor training and registration requirements in Australia and internationally.

2 Accurate feedback and evaluation are considered critical to effective supervision, yet supervisor evaluations are often posi- tively biased.

3 There are a range of barriers supervisors experience when pro- viding feedback and evaluation, and these may contribute to biased evaluations.

Clinical supervision is considered the cornerstone of psychol- ogy training (Falender & Shafranske, 2004) by both directors of clinical programmes and postgraduate students (Scott, Pachana, & Sofronoff, 2011). Recently, emphasis on competency-based models for practitioner training in psychol- ogy and other health disciplines has greatly increased the demands on the clinical supervisor to demonstrate, in an

Correspondence: Craig J Gonsalvez, School of Psychology, PG-87 Kingswood Campus, University of Western Sydney. Fax: 02 47360151; email: [email protected]

Accepted for publication 6 August 2015

doi:10.1111/ap.12175

What this paper adds

1 Provides a valuable comparison of goal-setting, feedback, and assessment practices conducted by clinical supervisors against best-practice guidelines for these supervisory tasks.

2 Highlights the need to promote the use of observational tech- niques (in vivo and video) and to support their implementation in day-to-day supervisory practice.

3 Highlights the need for measures to address supervisor leniency in summative assessments.

objective and transparent manner, that all supervisor respon- sibilities are discharged in accordance with evidence-based practice, pedagogic principles, and/or best-practice guidelines (Falender & Shafranske, 2014; Gonsalvez & Calvert, 2014; Kaslow et al., 2007; Milne, 2010).

In effect, although the roles of the clinical supervisor have not substantively changed, supervisor training and supervisory practices have been recently subjected to close pedagogic scru- tiny (Falender, Shafranske, & Ofek, 2014; Gonsalvez & Milne, 2010; Watkins & Milne, 2014). For instance, in the past, a few years of practitioner experience was deemed sufficient to become eligible to provide clinical supervision. The assumption that underpinned supervisor credentialing practices of the past, namely that “practitioner experience begets supervisory

Australian Psychologist 52 (2017) 21–30 © 2016 The Australian Psychological Society

21

Clinical supervision practices CJ Gonsalvez et al.

Australian Psychologist (2015) © 2015 The Australian Psychological Society

2

Clinical supervision practices CJ Gonsalvez et al.

expertise,” has been largely rejected (see Gonsalvez & Milne, 2010). In the UK, clinical supervision has been identified as an essential competency for psychologists (Gonsalvez & Milne, 2010); in Australia, supervisor accreditation has been made mandatory, and supervisor training programmes have been required to meet established guidelines to attain Board- approved provider status (Psychology Board of Australia, 2013).

The focus on competencies has highlighted the need to develop competency frameworks, articulate them into practice- based principles and implications, and evaluate outcomes for both the supervisee and the supervisor. Within this paradigm, supervisors themselves have to acquire a range of competencies in knowledge, skills, attitude value, and relationship domains, and to follow best-practice guidelines in planning and delivering supervision (Falender & Shafranske, 2004, 2014; Gonsalvez, 2014; Roth & Pilling, 2008). Thus, within a context of rapid change, a focus on core supervisory activities appears both timely and relevant for the Australian psychologist. The current study will focus on three activities—goal-setting, feedback, and assessment—that have been identified as important supervisor competencies (e.g., Psychology Board of Australia, 2013; Roth & Pilling, 2008).

Goal-setting

Effective goal-setting requires the planning and articulation of goals the supervisee should attain by the end of a placement. Although the importance of setting goals has been emphasised within supervision for a long time, recent developments have provided goal-setting with a theoretical framework and guiding principles to inform their implementation in supervisory practice (Gonsalvez & Calvert, 2014). Competency-based approaches recommend that supervision goals are best formulated as end- placement competencies (e.g., Falender & Shafranske, 2004) or as learning outcomes that are specific, measureable, appropriate from a developmental context, relevant, and time-wise—in other words, SMART (see Gonsalvez, 2014). There is broad agree- ment that supervision goals should take into consideration not only learning outcomes desired by supervisee and supervisor (Bernard & Goodyear, 2014; Milne, 2009; O’Donovan, Halford, & Walters, 2011), but also recommendations by training institu- tions, placement agencies, and professional and regulatory bodies such as the Australian Psychological Society and the Psychology Registration Board (Gonsalvez, 2014; Gonsalvez & Calvert, 2014). There is also broad consensus that the process of goal development should be planned and deliberate, should commence early in the supervision process, and be collabora- tive—with levels of supervisee engagement and autonomy being determined by the developmental/professional status of the supervisee (Falender & Shafranske, 2004; Gonsalvez, 2014; O’Donovan et al., 2011). Further, it is recommended that final goals should be comprehensive to ensure coverage of all impor- tant domains of competence and be documented in a supervisory contract (Bernard & Goodyear, 2014) or in a competency-based supervision plan (Gonsalvez, 2014).

Formative Feedback and Summative Assessment

Regular feedback informed by best-practice guidelines is a core supervisor competency and central to promoting students’

learning (Bernard & Goodyear, 2014; Cantillon & Sargeant, 2008; Robertson, Rosenthal, & Dawson, 1997). In the context of supervision, supervisor feedback can be defined as information provided to the trainee about their professional interactions and conduct to facilitate and support desirable progress and devel- opment (Archer, 2010).

Two different but complementary types of feedback and evaluation have been described: formative and summative (Bernard & Goodyear, 2014; Kealey, 2010). For the purposes of this report, we use the term “feedback” for formative comments, and “assessment” for summative aspects of evaluation. The term “evaluation” is used by some authors (e.g., Robiner, Saltzman, Hoberman, Semrud-Clikerman, & Schirvar, 1997) to refer to summative aspects, so this term is preserved in our measures derived from these authors and is used when citing their results or commentary. Formative feedback is idiographic in that it is supervisee-centred and is usually delivered through frequent qualitative comments during the entire course of a placement. Summative assessment is typically quantitative (e.g., scores on a Likert scale), occurs infrequently (e.g., at mid- and end- placement), and is based on predetermined criteria (Bernard & Goodyear, 2014; Kealey, 2010). The purpose of formative feed- back is to foster the attainment of competence by regular and ongoing monitoring of performance, and by providing relevant information to develop or enhance competencies across domains (e.g., assessment and intervention) and competency types (e.g., knowledge and skills). Summative assessment serves a gate-keeping purpose for the profession by ensuring that pre- determined standards are met before a trainee’s competence can be certified and the individual granted a licence to practice (Gonsalvez & McLeod, 2008; Milne, 2009). For placements, the “pass” standard for summative assessments can be normative (the supervisee’s performance is compared with peer-based norms) or criterion-based (the supervisee’s performance is com- pared with a predetermined standard of competence set by a regulatory authority or other stakeholder; Falender & Shafranske, 2004; Gonsalvez & Calvert, 2014; Tweed, Graber, & Wang, 2010). Because the outcome of feedback or assessment can be either negative (e.g., decreased performance) or positive (e.g., improved learning efficiency; Hattie & Timperley, 2007; Hoffman, Hill, Holmes, & Freitas, 2005; Kluger & DeNisi, 1996), a better understanding of factors that enhance feedback effec- tiveness will contribute to improved supervision outcomes.

Although our review of the literature did not yield empirical reports of effective versus ineffective feedback, the literature offers preliminary guidelines for the provision of feedback (e.g., Bernard & Goodyear, 2014; Milne, 2009) and for end-of- placement assessments (e.g., Gonsalvez & Crowe, 2014; Kaslow et al., 2007). For instance, Bernard and Goodyear (2014) have compiled a list of suggestions for the provision of feedback from a range of authors. Feedback should be “direct and clear, but never biased, hurtful, threatening, or humiliating,” must be “owned by the supervisor as professional perception, not fact or truth,” and “should address learning goals (competencies) that the supervisee can achieve (Bernard & Goodyear, 2014). Feed- back should be balanced between supportive and challenging information. Subjective feedback should be labelled as such and based on professional judgement rather than personal bias. Finally, feedback should be objective, behaviourally driven, and

Australian Psychologist 52 (2017) 21–30 © 2016 The Australian Psychological Society

22

CJ Gonsalvez et al. Clinical supervision practices

Australian Psychologist (2015) © 2015 The Australian Psychological Society

3

CJ Gonsalvez et al. Clinical supervision practices

as often as possible based on observation or representative samples of the trainees’ behaviour (Bernard & Goodyear, 2014; Newman, 2010). Feedback should elaborate on the potential strategies by which the student might improve performance (Corbett & Anderson, 2001; Shute, 2008). It should shift a supervisee’s attention to the task or to motivational processes (e.g., give specific examples to illustrate observations) rather than to the self (e.g., personal praise, discouraging feedback, feedback concerning student’s personality; Hattie & Timperley, 2007; Kluger & DeNisi, 1996).

Under circumstances when performance is poor, some researchers suggest that feedback should be discussed in self- referenced terms, as it improves expectations regarding future performance as well as increases a student’s belief that poor performance was due to effort and application (Kluger & DeNisi, 1996; McColskey & Leary, 1985).

Barriers to Feedback and Assessment

Although there is expert consensus that observational methods are central to informed and effective supervisor feedback (Bernard & Goodyear, 2014; Gonsalvez & McLeod, 2008; Reiser, 2014), survey results suggest that supervisory practices do not adhere to these recommendations. For instance, early results from clinical psychology supervisors and supervisees indicate that supervisors largely rely on students’ self-reports rather than in vivo/video observation of therapy sessions to monitor trainee performance (Gonsalvez, Oades, & Freestone, 2002). Data from cognitive behaviour therapists in the UK (Townend, Iannetta, & Freeston, 2002) and recent surveys with larger samples within Australia (Scott et al., 2011) largely confirm these findings. Without the benefit of in vivo or recorded observation of stu- dents’ clinical work, supervisors are understandably challenged when attempting to provide accurate feedback and assessments.

It has been argued that in order to provide feedback and evaluate students’ performance, supervisors should have an established knowledge of the standards of clinical competence (Ende, 1983; Gonsalvez & McLeod, 2008). Because many super- visors supervise only one or two students a year, their norma- tive referents may lack clarity and precision, a problem that may be worse for less experienced supervisors; O’Donovan et al., 2011). Moreover, it is unclear whether university personnel who may have access to such normative information regularly communicate this to supervisors. Other barriers that may hinder the provision of reliable feedback and assessment include a weak supervisory relationship, the supervisee’s lack of openness to feedback (Hoffman et al., 2005), supervisor concerns that feedback may damage the student’s self-esteem (Farnill, Gordon, & Sansom, 1997), and the supervisor’s discomfort pro- viding corrective feedback and dealing with potential negative supervisee reactions (Ende, 1983; Hoffman et al., 2005).

It is noteworthy that these concerns not only hinder supervi- sors from providing accurate and effective feedback but also contribute to biased evaluation (Gonsalvez & Freestone, 2007). In a study of supervisor bias in evaluation, 59% of supervisors admitted that their own ratings of students had been biased in terms of central tendency or leniency bias (Robiner et al., 1997). Two definition and measurement issues, namely the lack of objective measures of competence/incompetence and awareness

of inherent subjectivity in assessment, were rated as primary factors influencing supervisors’ biases. Interpersonal and per- sonal issues, particularly “difficulty providing negative assess- ments” and “guilt or fear about damaging a supervisee’s career or feeling responsible for lengthening or terminating their graduate education,” also contributed to leniency (Robiner et al., 1997). While trainees may respond to feedback and assessment with apprehension, delivery of effective feedback and assessment predicts a stronger supervisory relationship and greater trainee satisfaction with supervision (Lehrman-Waterman & Ladany, 2001).

Although the literature provides guidelines for effective feed- back, there is a dearth of systematic, empirical research on the topic. The aim of the current study is to provide a description of supervisors’ current goal-setting, feedback and assessment prac- tices, and the barriers associated with performing these tasks effectively.

Several hypotheses are posed: (1) Supervisee self-reports will be used more frequently than direct observation as a source to inform supervisor feedback. (2) Supervisors who receive relevant information from the universities regarding clinical standards will report less difficulty with making final judgements about a trainee’s competence. (3) Supervisors will rate leniency as the most prevalent bias in their own and other supervisors’ assessment of trainees. (4) Definition and measurement issues will be rated significantly higher than interpersonal and supervisor personal issues as factors that contribute to lenient assessments.

Methods

Participants

Participants were supervisors of postgraduate psychology train- ees in Australia. Initial contact was made using email addresses from the Psychology Board of Australia’s website (740 addresses). One hundred eight (14.6%) were returned with “invalid address” messages, leaving a participant pool of 632 individuals. All interested participants received a self-guided learning resource on supervision practices regardless of whether they decided to participate. Participants who consented to par- ticipate completed (15–30 min) the questionnaire anonymously online, and were given the opportunity to go in the draw to win a $400 gift voucher. Fully completed questionnaires were obtained from 113 participants, resulting in a response rate of 18%.

Participants’ mean age was 48 years (standard deviation [SD] = 11.23) and 68.1% were women. The sample of supervi- sors had considerable experience as psychologists (M = 18.81 years; SD = 9.55) and as supervisors (M = 11.35 years; SD = 8.29). Almost all (92%) had provided supervision within the last 12 months. Supervisors ranged widely with regard to how many students they supervise a year (range = 1–20, M = 4.43, SD = 4.66). All supervisors were registered psycholo- gists, holding a master’s or higher degree. The majority (97.3%) indicated they were psychology Board-approved supervisors and had completed an approved supervisor training programme (85%).

Australian Psychologist 52 (2017) 21–30 © 2016 The Australian Psychological Society

23

Clinical supervision practices CJ Gonsalvez et al.

Australian Psychologist (2015) © 2015 The Australian Psychological Society

4

Clinical supervision practices CJ Gonsalvez et al.

Measures

The online questionnaire was assembled for the purpose of this study and comprised a combination of items from prior research (n = 23) and new items (n = 53) designed for this study. Nine questions covered information about demographics and super- vision (e.g., years of experience as psychologist and supervisor).

Performance expectations and goal-setting

Sixteen items covered the areas of expectations, goal-setting, and the supervision contract (e.g., Have you ever received any written or verbal guide from the universities concerning the expected level of clinical and professional competencies trainees should reach by the end of the placement?). The goal-setting and supervision contract questions were derived from previous research and the supervision literature (Bernard & Goodyear, 2014; O’Donovan et al., 2011; Osborn & Davis, 1996). Examples are the following: “Do you establish a clear and specific super- vision contract or agreement that both you and your trainee sign?” (see Table 1).

Feedback

Twenty-nine items asked about feedback practices and were derived from the literature concerning (a) the different sources of information supervisors use to inform feedback (seven items, see Table 3)—the items were derived from Scott et al. (2011; e.g., How often do you use the following sources of information to inform the feedback you provide?); (b) the importance (nine items) and frequency (nine items) of the supervisor’s preferred feedback styles (see Table 4) with content derived from several authors (Bernard & Goodyear, 2014; Kluger & DeNisi, 1996; McColskey & Leary, 1985; Shute, 2008); and (c) supervisor concerns regarding feedback provision (four items) with item content derived from Farnill et al. (1997) and Hoffman et al. (2005).

Assessment

Eighteen items covered the area of summative assessment and were largely derived from Robiner et al. (1997) to examine supervisor beliefs about their own and peer biases affecting

Table 1 Means and Frequencies of Factors Influencing Supervision Goal

Planning

M SD Mean Endorsed

rank (%)†

Developmental stage of trainee 4.22 0.86 4.14a 96.4

Specific learning needs of trainee 4.20 0.75 4.31a 100.0

APS/PBA competencies 3.91 1.01 3.78b 91.1

Trainee’s expressed wishes 3.76 0.78 3.33b 96.5

Clinical needs of the service 3.37 1.14 2.87c 79.6

University programme objectives 3.25 1.07 2.57c 74.3

Notes. APS, Australian Psychological Society; PBA, Psychology Board of

Australia. 1 = Not at all; 2 = a little; 3 = quite a bit; 4 = a lot; 5 = very much

so. a,b,cMean ranks that do not share a letter differ from each other at

p < .01.†Percentages were based on scores greater than 3 (“quite a bit”).

assessment, the types of biases they believed affected ratings (leniency, central tendency, and strictness), and the factors con- tributing to supervisor biases (see Table 5).

Analyses and Results

Performance Expectations and Goal-setting

In response to the question “In terms of performance expecta- tions you have of your supervisees, where do you source this information?” 83% of supervisors endorsed “psychology profes- sions expectations,” 69% endorsed “experience with prior supervision of students,” and 59% endorsed “university expec- tations.” Approximately half (52%) of the supervisors indicated receiving verbal and written guidance from the university pro- grammes, 27% indicated they received either verbal or written guidance, and 20% received no guidance. Half (50%) of the supervisors indicated that the universities provided little or no information regarding their expectations of supervision.

Table 1 provides the data for various factors that influence goal planning. Supervisors reported that establishing goals and supervision contracts were common practices. Goals were based significantly more on the developmental stage and learning needs of the supervisee than the objectives of the placement provider or those of the training institution. Participants were also asked to rate on a 0- to 10-point scale (0 = all trainee and 10 = all supervisor) whether the supervisee or supervisor deter- mined goal development, for the first and last external place- ments. The results showed that trainees were significantly less involved than their supervisors in driving goal development on their first external placement (M = 5.68, SD = 1.66) compared with when they were on their final placement (M = 3.55, SD = 1.69), t(112) = 12.46, p < .001, d = 2.12.

Participants were asked to rate the frequency with which they engaged in seven goal-setting, feedback, and assessment prac- tices (see Table 2). Not only are goals and supervision contracts routinely developed, but providing feedback and completing assessments are both frequently practised. Explaining the process of evaluation was completed “all the time” by only 57% of supervisors. Less than 10% of supervisors “often” or “always” received feedback regarding a student’s performance from pre- vious placements.

Feedback

It was hypothesised that self-reports would be more frequently used than direct observation (Hypothesis 1). The results (Table 3) revealed that supervisee self-reports informed super- visor feedback significantly more frequently (M = 3.86, SD = .74) than did direct observation of supervisee’s clinical work (M = 3.48, SD = 1.22), t(112) = 2.63, p < .01, d = 0.38.

Supervisors’ ratings of the frequency and the perceived importance of nine different feedback strategies were generally consistent. The feedback strategies rated as important also tended to be provided more frequently, such that the rank order of importance and frequency for these different components were very similar (see Table 4).

Providing self-referenced feedback (i.e., comparing a supervisee’s performance with their own prior performance), as

Australian Psychologist 52 (2017) 21–30 © 2016 The Australian Psychological Society

24

CJ Gonsalvez et al. Clinical supervision practices

Australian Psychologist (2015) © 2015 The Australian Psychological Society

5

CJ Gonsalvez et al. Clinical supervision practices

Table 2 Supervisors (%) Who Endorsed the Following Goal-Setting, Feedback, and Assessment Practices

Item descriptions Never- Often All the

sometimes time

Set goals 4.4 18.6 77.0

Supervision contracts developed 12.4 12.4 75.2

Provide feedback regarding performance 3.5 57.5 38.9

Explain the process of evaluation 21.3 22.1 56.6

Trainee allowed time to self-reflect before feedback is provided 14.2 48.7 37.2

Face-to-face discussion of trainee’s evaluation 3.6 19.5 77.0

Evaluation regarding prior performance received from previous placements 90.3 8.8 0.9

Notes. 1 = Never; 2 = rarely; 3 = sometimes; 4 = often; 5 = all the time.

Table 3 Reported Frequency and Percentages With Which Supervisors Endorsed Using Different Sources of Information to Inform Their Feedback

Item descriptions Mean SD Mean rank Never Rarely Sometimes Often All the time

Supervisee self-report of their behaviours 3.86 0.74 4.86a 0.0 3.5 24.8 54.0 17.7

Supervisee self-report of client outcomes 3.75 0.86 4.67ab 0.9 7.1 25.7 48.7 17.7

Direct observation of the supervisee in session 3.48 1.22 4.32bc 8.0 14.2 23.0 31.9 23.0

Systematic assessment of client outcomes 3.32 1.01 3.98cd 2.7 18.6 36.3 29.2 13.3

Supervisor and supervisee co-therapy 3.12 1.19 3.68de 14.2 12.4 30.1 33.6 9.7

Report by others (e.g., co-therapy) 3.02 0.86 3.32e 4.4 21.2 43.4 30.1 0.9

Review of audio-taped or videotaped recording of session 2.80 1.47 3.16e 29.2 16.8 14.2 24.8 15.0

Notes. 1 = Never; 2 = rarely; 3 = sometimes; 4 = often; 5 = all the time. a,b,c,d,eMean ranks that do not share a letter differ from each other at p < .01.

Table 4 Importance and Frequency of Use of Different Types of Feedback

Item descriptions Frequencya Importanceb

M SD M SD

Providing balanced feedback 4.20 0.64 4.58 0.65

Providing feedback that is specific and directive 4.19 0.61 4.60 0.61

Providing feedback in a timely fashion 4.18 0.64 4.49 0.66

Providing objective feedback that is based on direct observation 4.02 0.78 4.47 0.58

Providing task-oriented feedback 4.02 0.65 4.29 0.61

Providing goal-oriented feedback 3.66 0.73 4.12 0.71

Providing general praise or criticism for personal characteristics of the student 3.59 0.79 3.92 0.81

Labelling subjective feedback as personal opinion 3.51 1.04 3.76 0.98

Providing self-referenced feedback 3.51 1.01 3.96 1.00

a1 = Never; 2 = rarely; 3 = sometimes; 4 = often; 5 = all the time. b1 = Extremely unimportant; 2 = unimportant; 3 = neither important or unimportant;

4 = important; 5 = extremely important.

well as labelling subjective feedback as personal opinion, was rated by supervisors as the least frequently used and less impor- tant feedback types and styles.

Issues identified as concerns by supervisors included “stu- dents’ lack of openness to feedback” (66%), “effects of feedback on the student’s self-esteem” (47%), and the “lack of direct observation” (46%). Less frequently endorsed concerns were “weak supervisory relationships” (17%) and “personal difficulty providing feedback” (10%).

Summative Assessment

Consistent with Hypothesis 2, supervisors who received rel- evant information from universities regarding clinical standards

(mean rank = 54.21, n = 90) r

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.