In this presentation a method for and results from evaluating a multi-site, after-school program will be presented. The focus is both on explaining a new, combination method we devised for estimating separate and joint effects of site and individual participants’ characteristics on program outcomes as well as on discussing assessment results. The new method we describe involves combining a regression discontinuity approach with contextual effects modeling procedures. The evaluation results discussed and approach are from an outcomes assessment of a six-site, after-school program known as “Houston’s Kids” implemented to address the needs of displaced and at-risk youth. Evaluation data were gathered over a two-year period from a true "panel" of program participants and control subjects. The program outcomes assessed included changes in students’ academic achievement (statewide test scores, grade changes, etc.) and in developmental assets. Results revealed differential effects of individual participant's socio-demographic characteristics on outcomes by program site and site characteristics (including the proportion of students who were economically disadvantaged; at-risk; English language proficient; residentially mobile; and who successfully passed the state’s mandatory reading test.
Prepared for presentation at Evaluation 2017, Washington, DC, 2017.Authors:
Angelica Adams, Communities in Schools of Houston; Heidi Brasher, YMCA of Greater Houston; Amy Corron, United Way of Greater Houston;
Emily Faron, United Way of Greater Houston; Karla Kessler, Alief Independent School District; Cheryl McCallum, Children’s Museum of Houston
Roger Durand, Presenter and Evaluator, Durand Research and Marketing Associates, LLC