Chapter One
Designing Interventions Integrating Assessment Paradigms
Reliable and valid assessment data must be collected prior to, during, and after treatment implementation to ensure that interventions are comprehensive and effective. In particular, decisions regarding intervention targets, as well as where, when, and how to intervene, are optimized when based on information regarding child, family, and school functioning. Over the years, several assessment and evaluation paradigms have been proposed to promote understanding and effective treatment for childhood disorders (Mash & Terdal, 1997). The major assessment models include (1) diagnostic/categorical, (2) dimensional or empirical, (3) functional, and (4) ecological paradigms. Each model has unique strengths and limitations, particularly regarding information provided to support effective treatment design.
The purpose of this chapter is to provide an overview of each of the four assessment models, with a particular emphasis on their relative advantages and disadvantages for the evaluation of children with physical and/or mental disorders. Because no single assessment paradigm provides sufficient information about child functioning, we discuss how data can be integrated across models such that one model's strengths counterbalance another's weaknesses. After data are gathered and integrated, one must be able to link the results of an assessment to treatment design and implementation. This linkage is discussed with a specific focus on functional assessment data. Once an intervention plan has been put into place, data continue to be gathered to document whether the intervention has been implemented as intended and whether desired outcomes have been obtained. Thus, we delineate how information collected within each of the four paradigms can contribute to intervention evaluation and ongoing modifications to treatment. Next, the use of these assessment models in home, school, and community settings is explicated by discussing practical issues and potential limitations. Finally, we present two cases to illustrate how assessment data are interpreted and integrated to design interventions for children with physical or mental disorders.
DESCRIPTION OF ASSESSMENT PARADIGMS
Several paradigms have been developed to facilitate the assessment of children's needs and to plan potentially effective intervention strategies. Each paradigm is based upon a unique set of assumptions and espouses the use of specific strategies to guide the assessment of child functioning. The following is a description of four paradigms that are commonly used in research and practice and have been demonstrated to be highly useful in understanding children's needs.
Categorical Assessment
Categorical assessment typically is used to determine whether an individual meets criteria for a particular diagnosis. This paradigm has been developed primarily within the field of medicine, but it has made a strong contribution to research and practice in other fields, including psychology. The criteria for making this decision generally are derived by a panel of experts after a thorough review of the research literature pertaining to the diagnostic condition. The criteria may be further shaped by field testing of provisional criteria to evaluate the ability of each proposed symptom to differentiate children known to have the disorder from those known not to have the disorder (e.g., see Frick et al., 1994). In addition, field testing can be useful in identifying the number of symptoms needed to optimize the prediction of clinically meaningful levels of impairment (e.g., see Lahey et al., 1994).
The fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994) is a categorical system widely used in the United States for determining whether individuals meet criteria for one or more mental health disorders (e.g., posttraumatic stress disorder, conduct disorder, autistic disorder). Clinicians make determinations about whether a diagnosis is present or absent based on criteria specified in this manual, which were derived by a panel of experts, primarily psychiatrists.
Diagnostic interviews commonly are used to assist clinicians in making categorical decisions about the presence or absence of mental health disorders. Interview procedures can vary with regard to their level of structure, but most are highly systematic and provide specific guidelines for requesting information and following up with probes. Diagnostic interviews differ with regard to their scope. Some interview techniques, such as the Autism Diagnostic Interview-Revised (Lord, Rutter, & LeCouteur, 1994), assess a narrow range of diagnostic entities. However, most diagnostic interviews examine a broad range of categories. Examples of some of the commonly used broadband, structured interviews are the Diagnostic Interview System for Children (DISC; Shaffer et al., 1996), the Diagnostic Interview for Children and Adolescents (DICA; Reich, Leacock, & Shanfeld, 1995), and the Kiddie-Schedule of Affective Disorders-Present and Lifetime Version (K-SADS-PL; Kaufman, Birmaher, Brent, Rao, & Ryan, 1996). Many structured interviews have been adapted for administration to parents as well as children.
The psychometric properties of diagnostic interviews vary greatly depending on the informant and the disorder being assessed. For children under the age of 14 years, the reliability and validity of interviews generally are more favorable when the informant is a parent as opposed to a child. Furthermore, parent-reported interviews generally are more psychometrically sound for externalizing as opposed to internalizing disorders (Schwab-Stone et al., 1996).
Categorical methods of assessment have some notable strengths. These methods can provide a framework for assessment by delineating the domains and specific symptoms that need to be evaluated. Categorical procedures help clinicians to organize clinical information, relate clinical findings to known and often validated patterns of behavior, explore potential correlates associated with diagnostic patterns, and predict developmental outcomes (Power & Eiraldi, 2000). Identifying the presence of a disorder can alert the clinician to a risk for serious functional impairment and can be useful in identifying potentially useful pharmacological and psychosocial treatments (see Phelps, Brown, & Power, 2002).
Categorical methods are also associated with several limitations. These methods identify the child as the source of the problem and fail to provide information about systemic factors that contribute to a child's difficulties (Power & DuPaul, 1996). Identifying merely the presence or absence of disorder does not account for gradations of symptom severity. Children may have subclinical levels of symptomatology, yet still encounter some level of impairment. Addressing the needs of these children was one of the main reasons the Diagnostic and Statistical Manual for Primary Care-Child and Adolescent Version (American Academy of Pediatrics, 1996) was created (Drotar, 1999). Of great concern is that categorical methods do not account for important differences in symptom presentation and severity across gender and developmental levels. Furthermore, categorical methods provide only limited information about how to design psychosocial and educational interventions (Gresham & Gansle, 1992).
Dimensional Assessment
The dimensional approach provides an assessment of domains related to various aspects of physical, cognitive, behavioral, emotional, and social functioning. These methods assess functioning along a continuum generally ranging from adaptive to nonadaptive, with no clear demarcation of the boundary between disordered and nondisordered (Achenbach & McConaughy, 1996). These approaches typically provide an assessment of the severity of functioning related to each dimension by comparing the child's level of functioning to that of peers of similar gender and age. Historically, the dimensions assessed using these methods have been derived empirically from parent, teacher, and self-ratings of behavior (Achenbach & McConaughy, 1996), although more recently, the dimensions are being derived on the basis of empirical as well as rational methods (Achenbach & Rescorla, 2001; Reynolds & Kamphaus, 1992).
Behavior rating scales frequently are used to provide a dimensional assessment of functioning. Rating scales can be differentiated with regard to their scope of assessment. Some rating scales are multiaxial and assess multiple domains of behavioral, emotional, social, and academic functioning. Examples of these types of measures, often referred to as broadband scales, are the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001), the Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1992), and the Devereux Scales of Mental Disorders (DSMD; Naglieri, LeBuffe, & Pfeiffer, 1994). Rating scales also have been developed to assess a narrow range of functioning. The number of narrowband constructs that have been assessed using rating scales is virtually limitless. Just a few examples of these measures are the ADHD Rating Scale-IV (DuPaul, Power, Anastopoulos, & Reid, 1998), the Social Skills Rating System (SSRS; Gresham & Elliott, 1990), and the Multidimensional Anxiety Scale for Children (March, 1997).
Behavior rating scales also differ with regard to the informant providing clinical information. Some rating scales have been developed for use with multiple informants, including the parent, teacher, and child (e.g., CBCL, BASC, SSRS). Also, procedures have been designed to assess peer-oriented social functioning through peer ratings (Asher & Dodge, 1986).
The psychometric properties of rating scales generally are strong. The construct validity of these measures often is supported through exploratory and confirmatory factor analyses. The reliability of the multiple subscales of these measures typically is demonstrated through indices of internal consistency and test-retest reliability. Furthermore, research regarding the association of these scales with measures of related constructs and the lack of association of these measures with scales of different constructs often is available to support the convergent and divergent validity of the instruments.
Because dimensional methods of assessment generally are norm-referenced, they are highly useful for determining the severity of symptom clusters. These approaches can provide information about how children function in real-life settings from multiple informants. Furthermore, rating scales can assess functioning across settings, in particular the home and school. Limitations of these approaches include their susceptibility to rater bias. Informants may vary greatly in the thresholds they apply in making determinations about whether a behavior is problematic (Reid & Maag, 1994). Also, rating scales typically are developed for use with the general population of children, and they may not be responsive to racial, ethnic, or cultural differences among children (Reid et al., 1998). Furthermore, dimensional approaches have limited treatment utility. These methods are helpful in understanding the range and severity of children's problems across settings, but they provide little information about the meaning and function of behavior, which can be highly useful in intervention design.
Functional Assessment
A number of terms have been used to describe functional assessment procedures (e.g., descriptive analysis, functional analysis, and functional behavioral assessment). In addition, problem-solving models of assessment (see Ikeda et al., 2002; Sheridan, Kratochwill, & Bergan, 1996) typically recommend the use of functional methods of assessment. The use of these various terms has led to some confusion as to what the term functional assessment actually means (see Haynes & O'Brien, 1990). For our purposes in this chapter, functional assessment refers to a broad set of procedures that examine the environmental events that maintain target behaviors (Horner, 1994). More specifically, functional assessment includes procedures that comprise a descriptive analysis (e.g., interviews, and direct observations of target behaviors and environmental events as they naturally occur), as well as those strategies used in an experimental analysis (e.g., systematic manipulation of environmental events to examine the functional relationship between the target behaviors and environmental events).
The goal of functional assessment is identifying environmental contingencies that maintain target behaviors (e.g., escape from anxiety-provoking medical procedures) as well as establishing operations (e.g., fatigue due to lack of sleep) and/or antecedent events (e.g., presentation of an academic task) that may set the occasion for target behaviors to occur. Information gathered during the functional assessment can be used to determine which interventions are most likely to be effective for a specific child within a specific setting. In fact, a rich literature base in the fields of developmental disabilities (Iwata et al., 1994) and behavior disorders (Kern, Childs, Dunlap, Clarke, & Falk, 1994) documents the efficacy of interventions designed using functional assessment data.
A comprehensive functional assessment includes three steps: (1) interviewing, (2) descriptive analysis, and (3) experimental analysis (for details of these steps, see Nelson, Roberts, & Smith, 1998). In most applied circumstances, a functional assessment involves interviews and the collection of descriptive data rather than direct manipulation of environmental contingencies by the assessor (see Nelson et al., 1998, for discussion of this issue). Although one cannot incontrovertibly determine function through a descriptive assessment, by conducting interviews with a variety of sources (e.g., student, parent, teacher, nursing staff) and observing the target behaviors in relation to environmental events, the topography of the behavior can be defined, and a potential function for the target behavior can be hypothesized (Nelson et al., 1998). Interventions can then incorporate the hypothesized function to change behavior in the desired direction. For example, if a student is hypothesized to engage in off-task behavior to gain peer attention (i.e., commonly referred to as "class clown" behavior), then the intervention should include attention from peers contingent on appropriate, on-task behavior (e.g., peer tutoring) (for more examples, see DuPaul & Ervin, 1996).
Functional assessment procedures have several positive features. Chief among these is that specific intervention strategies can be derived directly from these data (i.e., minimal inference is required).
Continues...
Excerpted from Promoting Children's Health by Thomas J. Power George J. DuPaul Edward S. Shapiro Anne E. Kazak Copyright © 2003 by The Guilford Press. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
