Assignment: Creating a Single-System (Subject) Design Study The steps at the heart of single-system (subject) research are par
Assignment: Creating a Single-System (Subject) Design Study
The steps at the heart of single-system (subject) research are part of the everyday practice of social work. Each day social workers implement interventions to meet clients’ needs and monitor results. However, conducting proper single-system (subject) research entails far more than these simple day-to-day practices. Proper single-system research requires a high degree of knowledge and commitment. Social workers must fully understand the purpose of single-system (subject) research and the variations of single-system (subject) design. They must develop a hypothesis based upon research and select the right design for testing it. They must ensure the reliability and validity of the data to be collected and know how to properly analyze and evaluate that data. This assignment asks you to rise to the challenge of creating a proposal for a single-subject research study.
To prepare for this Assignment, imagine that you are the social worker assigned to work with Paula Cortez (see the case study, “Social Work Research: Single Subject” in this week’s resources). After an initial assessment of her social, medical, and psychiatric problems, you develop a plan for intervention. You also develop a plan to monitor progress in your work with her using measures that can be evaluated in a single-system research design. As a scholar practitioner, you rely on research to help plan your intervention and your evaluation plan.
Complete the Cortez Family interactive media in this week’s resources. Conduct a literature search related to the chronic issues related to HIV/AIDS and bipolar mental disorder. Search for additional research related to assessing outcomes and theoretical frameworks appropriate for this client. For example, your search could include terms such as motivational interviewing and outcomes and goal-oriented practice and outcomes. You might also look at the NREPP database identified in Week 1, to search for interventions related to mental health and physical health.
By Day 7
Submit a 5- to 7-page proposal/research plan for single-system (subject) evaluation for your work with PAULA CORTEZ. Identify the problems that you will target and the outcomes you will measure, select an appropriate intervention or interventions (including length of time), and identify an appropriate evaluation plan. *PROPOSAL/RESEARCH PLAN SHOULD BE BASED ON PAULA CORTEZ*
Include a description of:
- The problem(s) that are the focus of treatment
- The intervention approach, including length of time, so that it can be replicated
- A summary of the literature that you reviewed that led you to select this intervention approach
- The purpose for conducting a single-system (subject) research evaluation
- The measures for evaluating the outcomes and observing change including:
- Evidence from your literature search about the nature of the measures
- The validity and reliability of the measures
- How baseline measures will be obtained
- How often follow-up measures will be administered
- The criteria that you would use to determine whether the intervention is effective
- How the periodic measurements could assist you in your ongoing work with Paula
Single-System Studies
Mark A. Mattaini
ocial work practice at all system levels involves action leading to behav- ioral or cul tural change. The primary role of social work research is to provide knowledge that contributes to such professional action. vVhile descriptive resea rch about human and cultural conditions, as discussed elsc·where in this volume, can be valuable fo r guiding professional acti on,
know ing how to most effectively support change is critical for practice. A central qu estion for social work research, therefore, is "what works" in practice, what works to address what goals and issues, with what populations, under wha t contextual conditions. While descriptive research ca11 suggest hypotheses, the only way to really determ in e howweU any fo rm of practice works is to test it, under the most rigorous conditions possible.
Experimen tal research is therefore criti cal for advancin g social work practice. Unfon·tunately, only a small proportion of soc ial work research is experimental (Thyer, 200 I). Experiment al research is of two types, group experiments (e.g., randomized clinical trials [RCTs]) and single-system research (SS R, also commonly referred to as single case resealfch, N of 1 research, o r interrupted time-series experiments). Si ngle-system experi- mental research, however, has often been un deremphasi zed in social work, in part because of limited understanding of the logic of natural science among social scientists and wcial workers.
SSR is experimental research; its purpose, as noted by Horner and colleagues (2005 ), is "to document causal, or functional, relationships between independent and dependent variables" (p. 166). The methodology has been used with all system levels-micro, mezzo, and macro-m aking it wid ely appl iCable for studying social work concerns. For example, Moore, Delaney, and Dixon (2007) studied ways to enha nce quality of life for quite impaired patients with Alzheimer's disease using singl e-system methods and were able to both individualize interven tions and produce generalizable knowledge from their study in ways that perhaps no other research strategy could equa l. In another example, Serna, Schumaker, Sherman, and Sheldon (1991) worked to improve family interactions in families with preteen and teenage children. The first several interven tions they attempted (interventions that are common in social work practice) fa iled to produce changes that generalized to homes. Single-system procedures, however, allowed them to rigorously and sequentially test multiple approaches until an adequately powerful intervention strategy was refi ned. (Note that Lhis would be impossible using group methods without under- mining the rigor of t he study.)
241
242 PART II • QUANTITATIV( APPROACHES: TYPES OF S TUDIES
Turning to larger systems, single-system designs can be used, for example, to examine the relative effects of different sets of organizational and community contex.'tS on the effectiveness of school violence p reven tion effo rts (Ma ttaini, 2006). Fu rthermore, Jason, Braciszewski, O lson, and Ferrari (2005) used multiple baseline single-system methods to test the impact of policy changes on the rate of opening mutual help recovery homes for substance abusers across entire states. Embry and colleagues (2007) used a similar design to test the impact of a statewide intervention to redu ce sales of tobacco to m in ors.
Although sin gle-system m e thods are widely used for practice monitoring in social work, research and monitoring are different endeavors with different purposes. This chapter focuses on the utility of SSR for knowledge building. Readers interested in 1 he use of single-system methods for p ractice mon itoring are likely to find Bloom, Fischer, and Orme (2006) and Nugent, Siep pert, and Hudso n (2001 ) particularly helpful.
Understanding Single-System Research
Single-system experimental research relies on natural science methodologies, while much of the rest of social work research, including a good deal of group experimental research, emphasizes social science methods. The differences are real and s ubstantive. In 1993, Johnston and Pennypacker noted,
The natural sciences have spawned technologies that have dramatica lly transformed the h u man culture, a nd the pace o f t echno logical development o nly seems to increase. The social sciences have yet to offer a single well-develop ed techno logy that has had a broad impact on daily life. (p. 6)
T here is li llie evidence that this s ituation has changed. The reasons involve bo th meth- ous and philosophies of science. Critically, however, analysis is central in most natural sci- ences and is best achieved through the direct manipulation of variables and observation of the impact of those manipulations over a period of time. As one expert noted, the heart or SSR is demonstrating influence b y "mak[ing] things go up an d down" under precisely specified conditions (J. Moore, personal communication, 1998) . Such analysis is often best done one case at a time.
SSR has particular strengths for social work research. SSR focuses on the individual sys- tem, the indiv id ual person, the individ ual family, and the individual neighborhood, typi- cally the level of analysis of primary interest in social work. Fur thermore, SSR allows detailed analysis of intervention outcomes for both responders and nonresponders, which is critical for practice because each client, not just the average client, must be of concern. Relevant variables can then be further manipulated to understand and assist those who have not responded to the in itial manipulations (Horner ct al., 2005). furthermore, as noted by Horner and colleagues (2005 ), rigorous SSR can be implemented in natural and near natural conditions, making it a practical strategy for elaborating and refining inter- ventions with immediate appl icabili ly in standard service setti ngs.
Contrasts With Group Experimental Research Most group exper imen tal research reli es on comparin g the impact of one or more inter- ventio ns (e.g., experimental treatment vs. standard care, placebo therapy. or no treat- ment) applied to more or less equivalent samples. Ideally, these samples are randomly
CHAPTER 14 • SJNGLE·SYSTfM RESEARCH 243
selected from a larger population of interest, but in social work research, it is more common for samples to be chosen on the basis of availability or convenience. Comparison studies include (a) classical experiments with randomization and no – intervention controls, (b) contrast studies that compare one intervention with another, and (c) a wide range of quasi-exper imental des igns. W"hilc comparison studies, espe- cially ra ndomized clinical trials, are often regarded as the gold standard for ex-perimen- tal research, the often unacknowledged strategic and tactical limits of !>uch comparison studies are serious (Johnston & Pennypacker, 1993, p. 119). Conclusions rely on proba bilistic meth ods drawn from the social sciences, rather than on the analytic methods of SSR. As a result, Jo hnston and Pennypacker ( 1993) suggest that comparison studies "often lead to inappropri~te inferences with poor ge nera li ty, based on improper evidence gath ered in support of the wrong question, thus wasting the field's limited experimental resources" (p. 120). (Similar criticisms have been made of much descrip- tive research.)
Wh ile comparison stu dies are useful for many purposes (as outlined elsewhere in this volume), it is important to understand their limits. As is true of mo st social science research, comparison studies at their core are actuarial. They attempt to determine which of two procedures produces better results on average (Johnston & Pennypacker, 1993 ). Jn pretty much all cases, however, some persons (or groups, organizations, or communi ties ) will do bcller, some will show m inimal change, an d others wil l do worse. Comparison studies by their nature do not provide in formation about the variables that may explain why these within -group differences occur; rather, such differences, while acknowledged, are generally trea ted as error. Analytic natural science methods, however, including rigor- ous SSR, can do so.
In addition,
although two procedures may address the same general behavioral goal, a number of detailed differences among them may often make each an inappropriate metric for the other. These differences may in clude (a) the exact characteristics of the populations and settings where ea ch works best, (b) the target behaviors and their controlling infl uences, or (c) a variety of more administrative considerations such as the characteristics of the personnel conducting each procedure. (Johnston & Pennypacker, 1993, p. 122)
Similar issues are present for large system work like that done in community practice and prevention science. Biglan, Ary, and Wagenaar (2000) note a n umber of limita tion!> lo the use of comparison studies in community research, including "(a) the high cost of research d ue to the number of communities needed in such studies, (b) the difficulty in developing generalizable theoretical principles about community change proccs. e through randomized trials, (c) the obscuring of relationships that are unique to a subset of communities, and (d) the problem of diffusion of intervention activities from in ter- vention to control communities" (p. 32) . SSR, particularly the use of sop histicated time- series designs with matched communities (Biglan et al., 2000; Coulton, 2005), provides powerful alternatives that do not suffer from these limitations.
Analytic investigations, in co ntrast to actuarial studies, allow the researcher to manip- ulate identified variables one at a time, oflen with one system at a time, to explore the impact of those variables and the d ifferences in such impacts across systems, as well as to test hypoth eses about the differences found. This is the natural science approach to inves- tigation, this is how generalizable theory is built, and this is primarily how scientific advance occurs. Kerlinger (1986) states, "The basic aim of science is theory. Perhaps less
244 PART II 8 Q uANTITATIVE APPROACH~S: T YPES Of S TUDIES
cryptically, the bas ic aim of science is to explain na tural phenomena" (p. 8). Social ·vork needs to be able to understand how personal and contextual factors important to client welfare and human rights can be influenced, and analytic studies are needed to move the field in that direction and thus "transform . .. human culture" (Johnston & Pennypacker, l ~93, p. 6 ). Once the relevant variables and contingent relatio nships have been clarified through analytic s tud ies, grou p experimental comparisons may have unique co ntribu- tions to m ake in organ izational cost-benefit comparisons and other areas as outl ined else where in this volume.
The Logic of Single-System Research The basic logic underlying SSR is straightforward. Data o n the behavior of interest are collected over a period of time until the baseline rate is dearly established. Intervention is then introduced as data continue to be collected. In more rigorous single-system studies, intervention is independently introduced at several points in time, while hold ing contex- tu al conditions co nstant, to confirm the presence of functional (causal) relationships. (Repeated measurement of the dependent variable[s] over time, therefore, is central to SSR.) As discussed later, a great deal is now kJ1own about how to achieve high levels of experimental control and validity in the use of these procedures.
Behaviors of interest in SSR may include those of individuals (clients, family members, service providers, policy makers) as well as aggregate behaviors am ong a group (students in a class, residents in a state). In addition , behavior as used here includes all ronns of actions in context (Lee, 1988), including motor behaviors (e.g., going to bed), visceral behaviors {e.g., bodily changes associated with emotions), verbal behaviors (e.g., speaking or covert self-talk), and observational behaviors (e.g., hearin g or dreaming).
A number of dimensions of behavior can be explored and pote ntially changed in SSR, including rate (frequency by uni t of time), in tensity, duration, and variability. Single- system researchers therefore can measure the impact of intervention (or prevention) on (a) how often something occurs (e.g., rate of suicide in a state), (b) how strongly it is present (e.g., level of stress), (c) how long something occurs (e.g., length of tantrums), and (d) how stable a phenom enon is (e.g., whether spikes in violence can be eliminated in a neighborhood). Nearly everything that social work research might be interested in, therefore, can be studied using SSR techniques, from a client's emotional state to rates of violations of human rights v.rithin a population.
Nearly all SSR designs depend on first establishing a stable baseline, the rate (or inten- sity, duration, variab ili ty) of behavior before intervention. Sin ce all behavior va ries to some extent over time, multiple o bservations are general ly necessary to establish the extent of natural va riability. In some cases, a baseline of as few as three data points may be adequate; in general, however, the more data points collected to establish baseline rates, the greater th e rigor of the stud y.
Once a stable baseline has been o btained, it is possible to introd uce a systematic varia- tion in conditions (i. e., an in ter vention, or one in a planned ser ies of intervent ions) and to determine whether that intervention is followed by a change in the behavior(s) of interest. The general standard for change in SSR is a shift in level, trend, or variability that is large, clearl y apparent, relatively immediate, and clinically substantive. (Technical details regarding how such changes can be assessed graphically and statistically are provided later in this chapter.) Fig ure 14.1 presents the most basic structure of the approach, depicting a clear change between phases. (Much more rigorous designs arc di scussed later in this chapter.)
Figure 14.1 A graph of data for a si mple single-system research design
with successive observations plotted on the horizontal axis and frequencies of a
behavior of interest on the vertical axis. (This graph depicts a n A-B [baseline- intervention] design, wh ich will be
di scussed in deta il later in the cha pter.)
CHAPTER 14 • S INGLE- SYSTEM R£S£ARtH 245
30 Baseline Intervention
25
1/) 20 Q)
'() c Q) 15 :J C" Q) … u. 10 ~—o
5
0 2 3 4 5 6 7 8 9 iO
Observations
Rigorous SSR requires strong measu rement, more complex designs comparin.; mclti- ple phases, and sophisticated analytic techniques. Horner and coll eagues (2005. Tab!e 1) identify a series of quality in di cators that can be used to judge the rigor of single- ;-.~tern invest igations, inc luding evaluation of descriptions and characteristics of particirants, descr iptions and characteristics of the setting, specification of independent and depen- dent variables, measureme nt procedures, esta blishment of experimental con~rol. and proced ures to ensure internal, external, and social validity. All of these dimensions l'l.;n be explored later in this chapter.
Two examp les of methodologically straightforward single-system studies illu- r .. ,e ihe co re logic of SS R. All day an d Pakurar (2007) tes ted the effec ts o r teacher greeting::. n rates of on-task behavior for three middle schoo l students who had been nominated o· their teachers for consistent difficully in remaining on task during the beginning o: t!le "-1lool day. Some existing research suggests that teacher greetings ma y have an impact on ::.:u<!em behavior and achievement (Embry, 2004); Allday and Pakurar wanted to e.:penrnmtally test this effect. They us ed a multiple baseline design (discussed in detail later . ~nning by co llec ting observational d ata in classrooms. After three observations, one ~eac~er hega n greeting the target st udent in her class with his name and a positive statement \-hen he entered the classroom. Meanwhile, the two other students, who were in JitTerem schoo ls, continued to be observed.
The rate of on-task behavior for Lhc first student immediately improved, , hile there was no change for the other two. Shortly thereafter, the first studcn l co n tinued ~o be greeted, the second student also began to be greeted, an d the third stud ent connnueci m just be o bserved. On-task behavior for the firs t student rem ained high and improved rub- stantially fo r the second, while there was no cha nge for the third. At the nex1: ob:>en"3tion point, greetings for the third student were added; at this point, the data for all iliree showed improvement over baseline. Each time the intervention was introduced. aac only when the intervention was introduced, th e dependent variable showed a ch …. rl;;e. Each time change occurred co ncurrent with intervention, the presence of a causal :elation beca me m ore convincing, the principle of unlikely successive coincidences (Thyer & Mvers, 2007) . I n addition, two o f the studen ts showed greater improvem ents than tile third.
246 PART II • QUANTITAIIVE APPROACHES: TYPES OF STUOIES
Those data indicate that the intervention tested was adequate for the first two students but that refinements may be needed for the third. This level of precision is critical for clin- ical research.
In a second example, Davis and colleagues (2008) reponed a single-system study with a 10-year-old boy who displayed multiple problem behaviors in the classroom that inter- fered with his own and others' Learning. After tracking his behaviors over a baseline period of 5 days, a social sk ills and self-control intervention was initiated. As soon as these procedures were implemented, the level of behavior problems dropped dramatically. When the procedures were withdrawn for 5 days, behavior problems rapidly increased again. When the procedures were reintroduced, behavior problems dropped 011ce more. The association between use of the intervention procedure and behavior problems becomes more persuasive each time rhey change in tan dem. Much more sophisticated and rigorous studies arc discussed below, some involving entire states in their sampling plans. What is important to note here, however, is the logic involved in demonstrating influence and control by introducing and withdrawing independent variables (interven tions) in planned ways to test: for functional relationships with dependent variables.
Rigor in SSR depends largely on two factors, the quality of the measurement used and the extent to which the design allows the investigator to rule out alternative explanations. In the Allday and Pakurar (2007) study, direct observation of the dependent var iable was imp lemen ted, with two observers used dur ing 20o/o of the observations. In the Davis et al. (2008) study, multiple measures, including direct onsite observation, were used (in lSo/o of observations, a second rater was used). In the Allday and Pakurar study, rigor was increased by introducing interventions one case at. a time to determine whether in terven- tion was fun ctionally related to behavior change. By con trast, stre ngthening rigor in the Davis et al. study involved introducin g and withdrawing procedures multiple tim es to determine whether presence or absence of the independent variable was consistently associated with behavior change.
Measurement in Single-System Research
There are a wide range of possible approaches for measuring independent and dependent variables in social work research. The most widely useful methods include direct observa- tion; self-monitoring by the client or research participant;· the use of scales, ratings, and standardized instruments completed by the client or other ra le rs; and the use of goal attainment scaling (GAS) or behaviorally anchored rating scales (BARS).
Observation Observation is the m ost direct and therefore often the most precise method of measuring behavior and behavior change. This is especially true when at least a sample of observations is conducted by more than one observc1·, which allows the calculation of i..nterobscrvcr reli- ability. Observation can be used to track such variables as the number of instances of self injury, the percentage of 10-second intervals in which a student is on task, repeated patterns that occur in family communication, or the immediate responses of decision makers to assertive behavior by clients participating in advocacy efforts, for example.
Observation often involves less subjective judgments, inferences, or estimates than other measures. For example, use of a rat ing scale related to the incidence of child behav- ior problems may involve judgments as to whether the rate is "high" or "very high," while
CHAPTER 14 • SiNGI F-SYSTEM RES£ARCH 247
a simple co unt prov ides both more precision and perhaps a less value -laden measure. There are times when direct observation is impractical, but given irs advantages, when- ever possible, it is the strategy of choice in SSR. The wide availability of video recording equipment has contributed to both the practicality of observation and the possibility of recordin g in the moment and analyzing later, anc.l it c~ n ~ lso facilitate measuring interob- server, or interrater, reliability. (Carefu l refinement and pretesting of operational defini – tions and training procedures should be built into observation planning, as the quality of obtained data may otherwise be compromised.)
There are times when observation is not practical due to cost, intrusiveness, or when rea ctivity to observation is likely to influence the behaviors oC inLcrcst. There also are tim es when observation and recording may rai se erhi ca l issues (as in some studies of ille- gal or antisocial behavior) . Some issues of social work concern are also not directly observable; emotional states and covert self-talk are examples. Other measurement app ro aches are needed under such circums tances.
Self-Monitoring Self-monitoring (self-observation) is a common and very useful approach for data collec- tion in social work SSR. It is often not possible for the researcher to "go home with the clie nt" to observe, for example, ch ild beh avior prob lems (althoug h sometimes this is in fact rea lisLic and usef-ul). From hundreds of studies, however, il is cl ear that parents can record many kinds of data quite accurately, from the frequency of tantrums or successful toileting to the extent to which they are frustrated with the child. Couples can monitor the number of ca ring act ions their partners take ove r Lh e course of a week (e.g., in Stuarfs [1 980] "caring days" procedures). Depressed individu als ca n tra ck their activities and le,– els of satisfaction on an hourly basis to prepare for behavioral activation procedures (Dimidjian et al., 2006; Mattaini, 1997). So long as the measurement procedures are clear and the participant has the capacity and motivation to complete them, self-monitoring can be both highly accurate and quite cost-effective. Simple charts that are clear and com- muni cative for those completing them are usually essential and should be provided. Asking people to devise their own charting system often will not produce quality data, but collaborating with clients or participants to custom ize recording char ts can work 'ery well (studies involving multiple clients or participants require uniformity of recording .
Self-monitoring can itself be motivating to clients and research participants, providing immediate feedback and often a sense of control over o ne's life (Kopp, 1993) . As a result, self-monitoring procedures are often reactive; monitoring by itse]f may change behavior, usually in the desirable direction. (A s imilar issue can arise with other forms of monitor- ing, but this is a particula r issue with self-monitoring.) This can be an advantage for in tervention, when the primary interest is in working toward the client's goals, but can complicate analysis in research sin ce record ing constitutes an additional active variable that needs to be taken into account in analysis. Often the best option when reacti vity may be a problem is to begin self-monitoring without the planned intervention and examine the resulting data over several measurement points. If the dependent variable shows improvement, monitoring alone shou ld be continued until a stable level is achieved before introducin g further experimental manipulation.
Rating Scales and Rapid Assessment Instruments When observation is not possible or p ractical, rating scales can be a useful alternative. Either the participant (cl ient) or ano ther person (e.g., a socia l worker or a parcn 1) can
248 PART I I • QuAtHITATIVE A PPROACHES: TYPF~ OF STUOI[S
complete such scales. Self- anchored scales, for example, are completed by the cl ient- for example, rating one's level of anxiety on a 0 to 100 scale. Such scales often have excellent psychometric properties (Nugent et al., 2001) and can often be completed very frequently, thus providing fine-grained data for ana lysis. Several such scales can be co mbined, as in Tuckma n's (1988) Mood Thermometers or Azrin, Naster, and Jones's ( 1973) Marital Happiness Scale, to provide a more complete, nuanced, and mul tid imensional picture o f personal or couple func ti oning. Clinicians can complete r ating scales (e.g., the Clinical Ra ting Scale for family assessment; Epstein, Baldwin, & Bishop, 1983), and parents can complete ratings on child behavior.
T hel'e are many sta ndardi zed sca les and rating sca les available; perhaps most useful for social work p ractice an d research are rapid assessmen t instrum en ts (l~Is). RAis are brief instruments that can be completed quickly and are designed to be completed often. As a result, the researcher (or clinician) can collect an adequate number of data points to care- fully track events in the case and thereby identify function al relal ionships. Please refer to Chapter 5 (this volum e) fo r more inform ation regarding such inst ruments.
Goal Attainment Scaling and Behaviorally Anchored Rating Scales GAS (Bloom et al., 2006; Kiresuk, Smith, & Cardillo, 1994 ) is a measu rement and mon i- toring approach for tracking pro gress, usually on more than one goal area at the same time, that has been used for practice and research at all system levels. GAS can be used to concurrently track multiple goal areas for a single client/participant system , while provid- ing an aggregate index of progress. Tn addition, if GAS is used with a client populatio n, the scores can be aggregated to m easure program outcomes (Kires uk e t al., 1994) .
GAS is organized around the Goal Attainment Fo ll ow-Up Gu ide, a graph ic device that lists five levels of goal attainment on the vertical dimension (from most unfavorable out- come thought likely to most favorable outcome thought likely) and m ultiple scales (goal a reas) with relative weights across the horizontal. Thi. produces a m atrix; the items in the m atrix are typically individ ually tailored to the case. The midclle level is the "expected level of success" for that scale within the timeframe specified. A scale (or depression for a case in which the initial scores over a baseline period ranged berween 3 1 and 49 (a clini- cally significant level of depression) on the Generalized Conten tment Scale (Hudson, 1982) might list an expected level of 20 to 29 (subclinical ), a less tha n expected level of 30 to -19 (no change), and a most un favorable level of 50 or greater. Two levels of greater than expected would also be identified. There might also be sca les for anxiety, ac tivity level, and quality of partner relationship on the same follow-up guide; depression could be weighted as twice as important as the other scales if that was determined to be the most important goal. Books li sting many possible scale items have been produced for GAS to assisl in preparation.
Form ul as for calculating and aggrega ti ng stand<lrd sco res on GAS guides are also avail- able, and GAS has been widely used for program evaluation and research {e.g., Fisher & Hardie, 2002; Newton, 2002). Any goal or issue that can be framed in terms of expected and less than expected levels of progress can be incorporated into GAS, if the analyst has adequate familiarit y with the substantive issue or goal.
BARS (Daniels, 2000; Mattaini, 2007) is a variat io n of goa l attainment scaling methods in which each level is specified in clea r and observable behavioral terms. BARS can, there- fore, combine the advantages of observations and ratings with those of GAS, allowing aggregalion of quite different measures for program evaluation, for example. At the same time, detailed analysis should primarily be done at the level of th e case.
CHAPTER 14 • SI NG LE·SYSTE1 R ESEARCH 249
Existing Data In many cases, the data needed to comp lete a single-system study are already being collected and need on ly to be accessed. This is particularly co mmon in community and policy-level studies. Fo r exam ple, if investigators are in terested in red ucing levels of d rug- related and vio lent crime in a neighborhood, as in a recent st udy by Swenson and colleagues in South Carolina, they will typically find that relevant data are collected and reported on a regular (often monthly) and relatively fi ne- grain ed basis (Swenson, Henggeler, Taylor, & Addison, 2005 ) . The investigators initiated combined multisys- temic therapy and neighborhood development initiatives, viewing the neighborhood as the single system. Usi ng routinely collected data, they discovered th at police calls for service in the neighborhood, once one of the highest crime areas in the state, had dropped by more than 80%.
lnterobserver Reliability vVhen behav ior is being d irectly observed and counted or when a variable is being rated by observers using some form of rating scale, it is often important to determine the objec- tivity of the measures reported. The mosl common approach used to do so is to measure the extent to which two or more observers see the same things happening or not happen- ing. This can be particularly important when observation involves some judgment: For exan1ple, «Was that, or was that not, an act of physical aggressio n as we have operationally defined it?" There are a number of ways of reporting interobserver agreement. One of the simples t and often the most useful is the calculation of percentages of in tervals in which the o bservers agree and d isagree on the occurrence of a behavior of interest (e.g., in how m any 1 0-second intervals was a child on task). (Sim ilar percentages can be calculated for d uration and frequency data.) In some cases, such percentages m ay be artificially high, as when the behavior of interest occurs in very few or in most in tervals. In such cases, statis- tical tools such as kappa can correct for levels of agreement e}..–pected by chance. There are also circumstances in which correlations or other indices of agreement may be useful; see Bloom et al. (2006
Collepals.com Plagiarism Free Papers
Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.
Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS
Why Hire Collepals.com writers to do your paper?
Quality- We are experienced and have access to ample research materials.
We write plagiarism Free Content
Confidential- We never share or sell your personal information to third parties.
Support-Chat with us today! We are always waiting to answer all your questions.