Implementation integrity of practice-based coaching: Preliminary results from the BEST in CLASS Efficacy Trial

/in /by
The purpose of this article is to describe the practice-based coaching model used in BEST in CLASS, a Tier-2 classroom-based intervention comprised of evidence-based instructional practices designed to prevent and ameliorate the chronic problem behaviors of young children at risk for the development of emotional/behavioral disorders. Following a description of the model, data from
year two of an ongoing 4-year randomized control trial are presented that describe (a) the amount (i.e., dosage) of coaching teachers received during BEST in CLASS implementation, (b) the integrity with which coaches implemented the BEST in CLASS coaching model, and (c) subsequent teacher implementation of the BEST in CLASS strategies. Forty-eight (23 BEST in CLASS; 25 comparison) teachers and ten coaches participated in this descriptive study. Data indicate that following coaches’ observations, teachers received approximately 30 min of
practice-based coaching during coaching meetings each week of implementation, and integrity data indicate that coaches implemented the critical coaching skills during the coaching meetings with integrity. Adherence data indicate that teachers in the treatment group increased their extensiveness of the use of BEST in CLASS practices at both post-treatment and 1-month follow-up compared to com-
parison teachers; competence data indicated that teachers in the treatment group increased the quality of delivery of practices at post-treatment compared to comparison teachers. Implications of these findings for both future research and practice-based coaching implementation are discussed.

Measuring Teacher Implementation of the BEST in CLASS Intervention Program and Corollary Child Outcomes

/in /by

This study is part of a larger randomized efficacy trail examining the impact of Behavioral, Emotional, and Social Training: Competent Learners Achieving School Success (BEST in CLASS), a Tier 2 intervention that targets the prevention of emotional/behavioral disorders in young, high risk children. In this investigation, we examined teachers’ implementation and maintenance of instructional practices in early childhood classrooms and the corollary relationships between teacher implementation of the specific instructional practices associated with BEST in CLASS and child engagement and problem behaviors. Fifty-three teachers (26 in treatment and 27 in comparison) and 130 preschool-aged children (66 in treatment and 64 in comparison) participated. Findings indicated that teachers’ who received training and coaching in the BEST in CLASS intervention increased their use of specific instructional practices in comparison with teachers in the control condition. In addition, children whose teachers received the BEST in CLASS intervention demonstrated increased engagement and a decrease in problem behaviors in comparison with those children who were in the control group. Positive teacher-child interactions increased and negative teacher-child interactions decreased in the intervention group in comparison with the control group. Results are discussed in relation to measuring teachers’ implementation of instructional practices and implementation science.

Measuring Implementation of Evidence-Based Programs Targeting Young Children At Risk for Emotional/Behavioral Disorders: Conceptual Issues and Recommendations

/in /by

Sutherland, McLeod, Conroy, & Cox (2013).

Young children with and at risk for emotional/behavioral disorders (EBD) present challenges for early childhood teachers. Evidence-based programs designed to address these young children’s behavior problems exist, but there are a number of barriers to implementing these programs in early childhood settings. Advancing the science of treatment integrity measurement can assist researchers and consumers interested in implementing evidence-based programs in early childhood classrooms. To provide guidance for researchers interested in assessing the integrity of implementation efforts, we describe a conceptual model of implementation of evidence-based programs designed to prevent EBD when applied in early childhood settings. Next, we describe steep that can be used to develop treatment integrity measures. Last, we discuss actors to consider when developing treatment integrity measures with specific emphasis on psychometrically strong measures that have maximum utility for implementation research in early childhood classrooms.

An Initial Evaluation of the Teacher−Child Interaction Direct Observation System: Measuring Teacher− Child Interaction Behaviors in Classroom Settings

/in /by

Sutherland, K.S., Conroy, M.A., Vo, A., Abrams, L., & Ogston, P. (2012)

Children who engage in chronic problem behavior can present challenges for their early childhood teachers, and are more likely to develop negative relationships with these and other adults with whom they interact. Unfortunately, children who develop negative relationships with their teachers early in their school careers are more likely to have later academic and behavioral problems than children who have positive, supportive relationships. The salience of interactions between teachers and young children highlights the need for interventions that target building positive teacher–child interactions and increase the likelihood that these interactions are developmentally appropriate and supportive of children’s emotional and behavioral growth, as well as assessment tools to evaluate the effectiveness of these interventions. The purpose of this article is to describe the Teacher–Child Interaction Direct Observation System (TCIDOS), an observation system designed to capture teacher and child behaviors during classroom interactions to evaluate the effectiveness of a classroom- based intervention targeting improvements in teacher–child interaction patterns. Following a description of our conceptual framework and the development of the TCIDOS, we discuss preliminary reliability findings and future research directions as well as challenges inherent to collecting direct observational data in classroom settings.

Measuring Teacher Implementation of the BEST in CLASS Intervention Program and Corollary Child Outcomes

/in /by

Conroy, M.A., Sutherland, K.S., Wilson, R., Martinez, J., Whalon, K.J., & Algina, J.J. (2013)

This study is part of a larger randomized efficacy trial examining the impact of BEST in CLASS, a tier 2 intervention. In this investigation, we examined teachers’ implementation and maintenance of instructional practices in early childhood classrooms and the corollary relationships between teacher implementation of the specific instructional practices associated with BEST in CLASS and child engagement and problem behaviors. 53 teachers (26 treatment and 27 comparison) and 130 preschool-aged children (66 treatment and 64 in comparison) participated. Findings indicated that teachers’ who received specific training and coaching in the BEST in CLASS intervention increased their use of specific instructional practices in comparison to teachers in the control condition. Additionally, children whose teachers’ received the BEST in CLASS intervention demonstrated increased engagement and a decrease in problem behaviors in comparison to those children who were in the control group. Positive teacher-child interactions increased and negative teacher-child interactions decreased in the intervention group in comparison to the control group. Results will be discussed in relation to measuring teachers’ implementation of instructional practices and implementation science.

Improving Interactions Between Teachers and Young Children with Problem Behavior: A Strengths-Based Approach

/in /by

Sutherland, K.S., Conroy, M., Abrams, L., & Vo, A. (2010)

Young children who exhibit problem behavior are at-risk for a host of negative developmental outcomes. Early intervention at multiple ecological levels is crucial to interrupt what can be a detrimental transactional cycle between children and their caregivers, including their teachers. In this article we promote a strengths-based approach to improving teacher-child interactions, which may serve as a protective factor for children with early-onset problem behavior in early childhood classrooms. Specifically, we describe how increases in targeted teacher behaviors can foster the desirable behavior of young children by capitalizing on skills the children already possess while enhancing those skills necessary for social competence and pre-academic success.

Preliminary Psychometric Properties of the BEST in CLASS Adherence and Competence Scale

/in /by

Sutherland, K.S., McLeod, B.D., Conroy, M.A. Abrams, L.M., & Smith, M.M (in press)

The measurement of treatment integrity is critical to evaluate the efficacy and effectiveness of evidence-based programs designed to improve the developmental outcomes of young children at-risk for emotional/behavioral disorders. Unfortunately, the science of treatment integrity measurement lags behind the development and evaluation of evidence-based programs for young, high-risk children. This article describes the development and preliminary psychometric properties of the BEST in CLASS Adherence and Competence Scale (BiCACS), designed to measure the adherence and competence of delivery of the BEST in CLASS prevention program. Independent observers coded videotaped (N=116) and live (N=289) observations of teachers delivering the BEST in CLASS program. The BiCACS showed good inter-rater reliability and analyses provided some support for the validity of the measure. Implications for future research and integrity measurement work are discussed.

BEST in CLASS: A Classroom-Based Model for Ameliorating Problem Behavior in Early Childhood Settings

/in /by

Vo, A.K., Sutherland, K.S., & Conroy, M.A. (2012)

As more young children enter school settings to attend early childhood programs, early childhood teachers and school psychologists have been charted with supporting a growing number of young children with chronic problem behaviors that put them at risk for the development of emotional/behavioral disorders (EBDs). There is a need for effective, feasible models that help school personnel address the needs of these young children within early childhood settings. This article describes Behavioral, Emotional, and Social Training: Competent Learners Achieving School Success (BEST in CLASS). BEST in CLASS is a classroom-based intervention designed to help early childhood teachers master effective instructional strategies for preventing and ameliorating problem behavior in young children at risk for EBDs through a teacher training and performance-based coaching model. The BEST in CLASS intervention model is described, and preliminary data are presented, followed by a discussion of implications for practice and future research directions.

Preventing and Ameliorating Young Children’s Chronic Problem Behaviors: An Ecological Classroom-Based Approach

/in /by

Conroy, M., Sutherland, K., Haydon, T., Stormont, M., & Harmon, J. (2008)

The number of young children who demonstrate chronic problem behaviors placing them at high risk for future development of emotional and behavioral disorders is increasing. These children’s problem behaviors often exist prior to entering school and become apparent as they interact with their parents at home. In fact, researchers have suggested that children who demonstrate chronic problem behaviors and their parents often end up developing well-established negative interaction patterns than can evolve into coercive relationships and persist upon entry into school. This article describes an ecological classroom-based approach, which emphasizes changing teacher-student interaction patterns as a means of preventing and possibly ameliorating coercive interaction patterns demonstrated by young children and their teachers. First, a brief overview of the current service delivery models and intervention programs addressing young children’s behavioral excesses is presented. Next, a description of the ecological classroom-based intervention model for addressing the behavioral needs of these children is described. This section includes the theoretical frameworks on which the model is based an an overview of model components. Additionally, the application of the model to a school-wide systems approach is explored. Finally, future research directions are discussed.

Early Childhood Teachers’ Use of Effective Instructional Practices and the Collateral Effects on Young Children’s Behavior

/in /by

Conroy, M.A., Sutherland, K.S., Vo, A.K., Carr, S., & Ogston, P.L. (2013)

This investigation examined the effects of a classroom-based intervention, Behavioral, Emotional, and Social Training: Competent Learners Achieving School Success (BEST in CLASS), on teacher behaviors and child outcomes in early childhood classrooms. First, we examined the effects of professional development training and practice-based coaching (including performance feedback) on teachers implementation and maintenance of the BEST in CLASS model practices. Next, we examined the effects of teachers’ implementation of these practices on young children’s engagement and problem behaviors. Using a descriptive non experimental design, 10 teachers and 19 children received the intervention. Findings indicated that teachers’ use of the BEST in CLASS practices including rules, pre correction, opportunities to respond, behavior-specific praise, and instructive and corrective feedback increased from baseline to completion of the intervention and these increases maintained. In addition, children’s engagement increased while their problem behaviors decreased. Although these results are promising, the current investigation has limitations and the results should be viewed with caution.