The three documents included here for use at the 2009 AEA conference are: 1) a description of our paper, 2) PowerPoint slides for our presentation, and 3) tables demonstrating use of the Partial Comparisons Approach in evaluating two STEM projects in Washington State. #EvaluationUse #Prek...
3 attachments
Focus Search
- - + + Table 4 Partial Comparisons Approach for the Acceso la Ciencia, Access Science, Informal Science Education Project funded by NSF Exhibit Tables Handbooks/ Training Manuals Field Trips (Crops, Darigold Plant, PSC) Science Shows Internship Program Parent Involvement STEM Career Workshops & College Preparation Assistance Outcomes-Only Comparisons The program performed better than at earlier time (pre-post). + + + + + + + The program performed better than another program (cross-section). - - - - - - - The program performed better than broader group of programs (cross-section). - - - - - - - The program’s performance trend is in desired direction (time series). + + + + + + + Outcomes appear faster or better than expected. - - - - - - - Outcomes exceed initial goals or objectives. + - + - - + + Outcomes exceed established standards. - - - - - - - Process-Only Comparisons The program implemented a new set of activities, not previously conducted. - + - + - - + The program improved an existing set of activities. + + - + + + + The program staff can describe how the program differs from previous policy or practice. - - - - - - - Table 4, continued Partial Comparisons Approach for the Acceso la Ciencia, Access Science, Informal Science Education Project funded by NSF Exhibit Tables Handbooks/ Training Manuals Field Trips (Crops, Darigold Plant, PSC) Science Shows Internship Program Parent Involvement STEM Career Workshops & College Preparation Assistance Causal Interpretation The program staff can provide a compelling explanation for a documentable chain of events. - - - - - - - External observers can provide a compelling explanation for a documentable chain of events. - - - - - - - A key informant (insider) can provide a compelling explanation for a documentable chain of events. - - - - - - - The pattern of outcomes is uniquely related to the program. + + + + - - - The intervention is uniquely related to some infrastructure, in turn related to the outcomes. - - - - - - - Rival Interpretations The program staff can provide rationale for rejecting explanations (general climate and competing programs). + + + + + + + External observers can provide rationale for rejecting explanations (general climate and competing programs). + + + + + + + A key informant can provide rationale for rejecting explanations (general climate and competing programs). - - - - - - - A pattern of outcomes can provide rationale for rejecting explanations (general climate and competing programs). + + - - + + + Table 4, continued Partial Comparisons Approach for the Acceso la Ciencia, Access Science, Informal Science Education Project funded by NSF Exhibit Tables Handbooks/ Training Manuals Field Trips (Crops, Darigold Plant, PSC) Science Shows Internship Program Parent Involvement STEM Career Workshops & College Preparation Assistance Rival Interpretations Infrastructure can provide rationale for rejecting explanations (general climate and competing programs). - - - - - - - Policy Analyses Magnitude of positive outcomes far outweighs costs of program. + + + + + + + Outcomes achieved for the first time in this program. + + + + + + + Outcomes generate support for further desirable action. + + + + + + +Evaluating Impact: Use of Yin’s Partial Comparisons Case Study Approach Michael S