Effectiveness of Intensive Behavioral Intervention in a large, community-based program

https://doi.org/10.1016/j.rasd.2008.01.002Get rights and content

Abstract

Although the efficacy of Intensive Behavioral Intervention (IBI) for young children with Autism Spectrum Disorders (ASD) has been well documented in small model programs, IBI's effectiveness (i.e., does it work in the “real world”?) has been less studied and may not be as impressive, e.g. Bibby, Eikeseth, Martin, Mudford, and Reeves (2002). This study reports on the outcomes of 332 children, aged 2–7 years, enrolled in a large, community-based, publicly funded IBI Program in Ontario, Canada. File review data at intake and exit were compared on a group basis as well as on an individual basis. Results indicated statistically significant and clinically significant improvements: reduction in autism severity, gains in cognitive and adaptive levels, as well as a doubling of children's rate of development. There was a considerable heterogeneity in outcome, as expected, and children were classified into seven categories of progress/outcome. The majority of children (75%) showed some gains during IBI and 11% achieved average functioning. Although the study has clear methodological limitations (chiefly the lack of a comparison group), it suggests that IBI can be implemented effectively in a large, community-based program.

Introduction

Autism is defined by severe difficulties in three areas: reciprocal social relationships, verbal and nonverbal communication, and unusual repetitive behavior and restricted interests (APA, 1994, APA, 2000). Autism is co-morbid with intellectual disability in approximately 75–80% of cases (APA, 1994; National Research Council [NRC], 2001), although persons within the broader category known as Autism Spectrum Disorders (ASD) may have milder or no intellectual delays (Fombonne, 2005). Individuals with autism may exhibit severe behavior problems (e.g., self injury, aggression) and/or mental health problems, as well. An enormous variety of treatments have been proposed for autism (approximately 50 reviewed by Perry & Condillac, 2003), including some which are highly sensationalized in the media and via the Internet, but which are eventually demonstrated through research to be of limited or no benefit or, indeed, to pose significant risk of harm (e.g., Bebko, Perry, & Bryson, 1996; Dunn Geier et al., 2000).

Early, intensive treatment using behaviorally based methods, sometimes known as Intensive Behavioral Intervention (IBI), has a much stronger empirical basis than virtually any other intervention used with children with autism and is considered “best practice” for young children with autism, based on reviews of the research literature and the recommendations of several professional consensus panels, convened by the New York State Department of Health (NYSDOH, 1999), the U.S. National Institutes of Health in 2000 (e.g., Schreibman, 2000), and the U.S. Department of Special Education via the National Research Council (NRC, 2001). Most of these IBI studies are based on small samples (fewer than 30 children) in private or University-affiliated, model programs. See Handleman and Harris (2001) for detailed descriptions of some of these programs. Although they vary in a number of ways, there are certain commonalities among them: (1) early age at onset of treatment (usually before age 4); (2) a large quantity of intervention (typically 25–40 h/week for 1–3 years); (3) a curriculum which is comprehensive in scope, developmental in sequence, individualized for the child, and builds in generalization; (4) a positive-oriented, functional approach to problem behavior; (5) a highly structured and predictable learning environment; (6) highly trained and intensively supervised staff; (7) parent involvement (Anderson & Romancyzk, 1999; Dawson & Osterling, 1997; Green, 1996, Perry, 2002, Powers, 1992).

Psychosocial treatment research in the evidence-based practices field makes an important distinction between efficacy and effectiveness (e.g., Weisz, Jensen, & McLeod, 2005). Efficacy refers to whether a treatment or intervention can be shown to work under “ideal” conditions. Clients are selected to be “good candidates” with no co-morbidities; therapists are well trained (often graduate students) and well supervised; and treatment is carefully planned and implemented faithfully according to a manual. Effectiveness, on the other hand, involves whether the intervention actually works when applied in “real life” situations, where there is typically less control over relevant variables. Clients are likely to be more severe, complex, and/or heterogeneous; staff are likely to be less well trained and supervised; and treatment is likely to become diluted and to “drift” from the manual.

Applying this distinction to the IBI literature, we suggest that the efficacy of high-intensity IBI (e.g., 25–40 h/week) has been demonstrated and replicated, in small model programs, relative to: (a) low intensity IBI control groups (Lovaas, 1987; McEachin, Smith, & Lovaas, 1993; Smith, Groen, & Wynn, 2000); (b) “eclectic” community services (Cohen, Amerine-Dickens, & Smith, 2006; Howard, Sparkman, Cohen, Green, & Stanislaw, 2005); and (c) equivalent amounts of high quality special education (Eikeseth et al., 2002, Eikeseth et al., 2007; Howard et al., 2005), indicating that it is not simply the intensity but the behavioral nature of the intervention which is efficacious. In addition, there are a number of uncontrolled studies which have done simple pre–post (e.g., Ben-Itzchak & Zachor, 2007; Weiss, 1999) or within-group analyses which shed additional light on particular questions such as the importance of age and IQ (Fenske, Zalenski, Krantz, & McClannahan, 1985; Harris & Handleman, 2000) or the model of service delivery being parent- or clinic-directed (e.g., Sallows & Graupner, 2005).

Documentation of efficacy in these studies typically involves substantial gains (e.g., 10-point change in group means) on standardized measures of cognitive level, adaptive skills, language level, and diagnostic and/or behavioral severity. However, statistically significant changes in group means (or lack thereof) tell us very little about meaningful change at an individual child level. Clinical significance (Kazdin, 2005), on the other hand, refers to meaningful degree of change in an individual child (such as changing from the clinical to the average range, or making a 20-point gain on a standardized measure). In these studies, 40–50% of these children have “best outcomes” (usually defined as functioning in the average range on standardized measures, loss of autism and/or intellectual disability diagnosis, and ability to function in regular school classrooms without support). However, it is important to remember the outcomes are still highly variable, not all children achieve “best outcomes”, and most studies do not have long-term follow-up data (the notable exception being McEachin et al., 1993).

Although the body of efficacy studies is quite compelling, there are several issues regarding the choice of dependent variables, which typically focus on cognitive and adaptive skills (which are generic to developmental disability) rather than autism symptomatology or problematic behavior (Matson, 2007). In addition, the independent variable, i.e., the specific nature of the intervention, is not well specified in most studies (Lechago & Carr, in press).

Furthermore, the vast majority of children now receiving IBI throughout the world are not enrolled in one of these model programs and surprisingly little is known about the basic characteristics of the intervention they are receiving (Carr, 2006), let alone the quality (Perry, Prichard, & Penn, 2006) or effectiveness of IBI provided in community settings with all the associated constraints (lack of public funding in some places, and substantial challenges involved in hiring, training, and supervision, ethical requirement to treat children not fitting the profile of the model programs, etc.). Jacobson (2000) referred to these issues as a “profound threat to effectiveness”. The empirical demonstration of effectiveness requires documenting whether and to what extent a treatment is effective with less-than-ideal clients, less-than-ideal staffing and supervision procedures, and less than ideal treatment integrity.

Only a few effectiveness studies of IBI have been published and they tend to use weaker methodology, rendering conclusions difficult. Also, they tend to suggest outcomes which are much less favourable than those reported in efficacy studies from model programs, a typical finding in clinical effectiveness research (Kazdin, 2005). For example, Boyd and Corley (2001) conducted a retrospective file review and parent survey regarding 22 children who had received 9–36 months of IBI (prescribed to be 30 h/week), most of whom started before age 4. No IQ or other standardized assessment data were available. Based on file review and observation, 7 children (32%) were judged to be of average intelligence, but children continued to have significant rates of maladaptive behavior, none was fully integrated without support and none was judged to be similar to Lovaas’ “best outcome”. These children may have been lower functioning and older than those in Lovaas (1987), which is used as the “gold standard”. Bibby, Eikeseth, Martin, Mudford, and Reeves (2002) examined progress and outcomes in a total of 66 children in self-funded, parent-managed IBI programs in the U.K. Results indicated essentially no group mean change in IQ, significant but modest gains in age equivalents in adaptive behavior, and significant improvement in ratings of speech as well as behavioral pathology. A minority of children made “notable” gains of 15 points in IQ (27%) and adaptive behavior (33%). Only 6% approximated Lovaas’ “best outcome” children with average IQ and regular class placement without support. Compared to Lovaas’ (1987) this study included some children (16%) who were lower functioning and many (57%) who were older at intake. Also, programs were home-based and parent-managed, and treatment quantity (14–40 h/week) and quality was likely less optimal. The majority of the 25 different supervisors/consultants would not have met UCLA standards and they saw children, on average, only every 3 months. Sheinkopf and Siegel (1998) reported the only controlled effectiveness study, in which they compared the outcomes of 11 preschool children (retrospectively) whose parents reported doing an IBI home program (overseen by one of three supervisors in the community, not affiliated with any academic program and independent of the authors) to a matched comparison group (matched on CA, Mental Age (MA), sex, diagnosis, and time interval). After a mean duration of 16 months, with a reported mean of 19 h/week of IBI (plus 7 h/week at school), the IBI group had better outcomes than the control group who received 11 h of school. Although equivalent on pre-treatment IQ, there were large differences in IQ following treatment (about 25 IQ points) and modest differences in autism symptomatology, although most retained their diagnosis. This study is the best designed of the effectiveness studies and also reports the strongest results.

In summary, although IBI can be considered to be an efficacious treatment, there is, as of yet, little or no good evidence of effectiveness in applications outside small model programs. Large programs run in the community have not been evaluated and reported in the literature. The data reported in the present study provide important evidence as to the question of effectiveness in a large-scale, community-based, publicly funded IBI program.

In 1999, Ontario launched a province-wide IBI initiative which was based on research evidence and stakeholder consultation (MCSS, 2000; see Perry, 2002 for background). It is, to our knowledge, the largest and most comprehensive IBI program ever mounted in the world. IBI is provided free of charge (funded by the provincial Ministry of Children and Youth Services) by one of nine regional programs, each of which may have several subcontractors or partner agencies serving different geographic areas. Altogether, there are about 20 agencies involved, including children's mental health centres, centres for children with physical and/or developmental disabilities, and children's hospitals. Regions vary in size from large rural areas with few children to large urban areas serving hundreds of children. Bilingual services (English and French) are mandated in some jurisdictions. Approximately 800 children would have been in service at once during the period the study data were collected and there were roughly the same number of clinical staff (the program has since expanded again). Although there is an option for families to purchase approved services privately, the vast majority of children in the current study received services directly through the regional programs.

Apart from the size and scope of the program, there are three major differences between the Ontario IBI program and the published efficacy studies, which may impact upon the results of the current study. First, there have been tremendous capacity-building challenges in recruiting, training, supervising, and retaining large numbers of staff in a short time and this likely has implications for treatment fidelity and quality. Second, the children are not selected in a manner that may bias in favour of good results. In fact, the opposite may be the case since the Ontario IBI program is intended for more severe and needy children (children are not excluded for co-morbid diagnoses or low cognitive level as in most studies in the literature), and many children are older at intake than those in the efficacy studies. Third, parents are not selected in such a way as to have financial, intellectual, or personal resources which might be associated with better outcomes, as in some of the model programs. Since this was a file review study and parents did not have to consent to participate, the usual volunteer sample biases do not apply. Families in the Ontario program represent enormous diversity socioeconomically, linguistically, culturally, and psychosocially and the program has been implemented in a climate of considerable parent and political pressure, advocacy, media attention, and litigation. Taken together, these three factors might be expected to have the effect of making children's outcomes less favourable than those reported in the efficacy literature. Nevertheless, these conditions represent the reality in which IBI is being implemented in Ontario (and possibly elsewhere) and the conditions under which it is crucial that IBI's effectiveness be examined.

Thus, the purpose of this retrospective study was to describe the outcomes of children participating in the Ontario program, to address two main research questions. Question 1 was: Did children show statistically significant and clinically significant changes on developmental and diagnostic measures from the time they entered the program to the time they were discharged? In particular, did severity of autism, cognitive level, adaptive level, and rate of development change during children's participation in the IBI program? Given the heterogeneity in outcomes even in efficacy studies and the factors noted above which likely bias against good outcomes, Question 2 was: What was the range of progress/outcomes seen at exit? and, in particular, did any children resemble the “best outcomes” reported in the literature?

Section snippets

Participants

Psychological assessment file data were available for a total of 332 children with an entry assessment (within 3 months) and another assessment (usually at exit) served between 2000 (when the program began) until early 2006 when the study data were collected. This number constitutes all available files from all nine programs and represents approximately one-third of the entire population of children served in the program during that period. Start-up pressures and ongoing resourcing difficulties

Results

Comparisons of children's scores at intake and exit were performed for the three subgroups using SPSS GLM Repeated Measures Multivariate Analyses of Variance with group as the between factor and time as the within subjects repeated measure. Main effects of time, as well as interactions of group × time are reported (main effects of group are not reported here since they were considered earlier in Table 2).

Discussion

We have reported here on a large effectiveness study of IBI, examining children's outcomes in the Ontario IBI Program. Results indicated that children showed statistically significant and clinically significant reduction in autism symptom severity during the time they were involved in the IBI program. Cognitive level improved significantly for children, in some cases dramatically so. Children gained significantly in developmental skills (increased age equivalents) in all areas of adaptive

Acknowledgements

We are grateful to the Ontario Ministry of Children and Youth Services for funding this study. However, the views expressed here are those of the authors and do not necessarily represent the position of the Ministry. We appreciate the assistance of Alissa Levy, Helen Penn Flanagan, Alice Prichard, Abbie Solish, April Sullivan, and Kerry Wells with data collection, data entry and verification and Don Downer for data verification, scheduling, and other administrative support.

References (63)

  • A.S. Bondy et al.

    A picture's worth: PECS and other visual communication strategies in autism

    (2001)
  • R.D. Boyd et al.

    Outcome survey of early intensive behavioral intervention for young children with autism in a community setting

    Autism

    (2001)
  • Carr, J. E. (2006, November). Early and intensive behavioural intervention for autism: A survey of intervention...
  • H. Cohen et al.

    Early intensive behavioral treatment: Replication of the UCLA model in a community setting

    Developmental and Behavioral Pediatrics

    (2006)
  • G. Dawson et al.

    Early intervention in autism

  • J. Dunn Geier et al.

    Effect of secretin on children with autism: A randomized controlled trial

    Developmental Medicine and Child Neurology

    (2000)
  • S. Eikeseth et al.

    Intensive behavioral treatment at school for 4–7-year-old children with autism: A 1-year comparison controlled study

    Behavior Modification

    (2002)
  • S. Eikeseth et al.

    Outcome for children with autism who began intensive behavioral treatment between ages 4 and 7: A comparison controlled study

    Behavior Modification

    (2007)
  • P.A. Filipek et al.

    The screening and diagnosis of autism spectrum disorders

    Journal of Autism and Developmental Disorders

    (1999)
  • E. Fombonne

    The changing epidemiology of autism

    Journal of Applied Research in Intellectual Disabilities

    (2005)
  • R.P. Goin-Kochel et al.

    Early responsiveness to intensive behavioral intervention predicts outcomes among preschool children with autism

    International Journal of Disability, Development and Education

    (2007)
  • G. Green

    Early behavioral intervention for autism: What does the research tell us?

  • S.L. Harris et al.

    Age and IQ at intake as predictors of placement for young children with autism: A 4 to 6 year followup

    Journal of Autism and Developmental Disorders

    (2000)
  • J.W. Jacobson

    Early intensive behavioral intervention: Emergence of a consumer-driven model

    The Behavior Analyst

    (2000)
  • N. Johnson-Martin et al.

    The Carolina curriculum for preschoolers with special needs

    (1990)
  • A.E. Kazdin

    Clinical significance: Measuring whether interventions make a difference

  • R. Leaf et al.

    A work in progress: Behavior management strategies and a curriculum for intensive behavioural treatment of autism

    (1999)
  • Lechago, S. A., & Carr, J. E. (in press). Recommendations for reporting independent variables in outcome studies of...
  • O.I. Lovaas

    Behavioral treatment and normal educational and intellectual functioning in young autistic children

    Journal of Consulting and Clinical Psychology

    (1987)
  • O.I. Lovaas

    Teaching individuals with developmental delays: Basic intervention techniques

    (2003)
  • Cited by (96)

    View all citing articles on Scopus

    Portions of this study were presented in poster or presentation formats at: the Association for Behavior Analysis in May 2007 in San Diego, CA; the Society for Research in Child Development in April 2007 in Boston, MA; the Ontario Association on Developmental Disabilities in April 2007 in Barrie, ON; the Association on Behavior Analysis Autism Conference in February 2007 in Boston, MA; and at the Ontario Association for Behavior Analysis in November 2006 in Markham, ON.

    1

    Current address: Central East Autism Intervention Program, Kinark, Markham, ON, Canada.

    2

    Current address: Autism Intervention Program - Eastern Ontario, Children's Hospital of Eastern Ontario, Ottawa, ON, Canada.

    3

    Current address: Toronto Partnership for Autism Service, Surrey Place Centre, Toronto, ON, Canada.

    4

    Current address: South East Region Autism Intervention Program, Pathways for Children and Youth, Kingston, ON, Canada.

    5

    Current address: Private Practice, London, ON, Canada.

    6

    Current address: North Region Autism Intervention Program Child Care Resources, Sudbury, ON, Canada.

    7

    Current address: Hamilton-Niagara Autism Intervention Program Hamilton, ON, Canada.

    8

    Current address: Central West Autism Intervention Program, ErinoakKids, Mississauga, ON, Canada.

    View full text