Welcome to the Design & Analysis of Experiments TIG

The purpose of the Design & Analysis of Experiments TIG is to provide an active forum for AEA members who specialize in or have special interest in experimental evaluation research to connect and advance the methods and practice of this segment of the evaluation field.  Evaluations that use an experimental design (with randomization of treatment and control units) are distinctive and involve their own methods, practices and analyses, as well as many that connect to other TIGs’ foci.  We hope to engage members from other relevant TIGs in order to have cross-fertilizing discussions. 


Specific areas of scholarly discussion for this TIG include:

·      integrating randomization into program operations

·      how to analyze experimental data (ITT, ATE, TOT, LATE, CACE, moderator effects)

·      design strategies for getting inside the black box

·      analytic strategies for getting inside the black box

·      opportunities to explore mediation with experimental data

·      power analyses/considerations (e.g., re: differential effects, in clustered-designs)

·      opportunities/challenges from multi-site experiments (when pooling is/is not justified; ICCs and power implications; extending multi-level modeling to experimental data)

·      within study comparisons (under what conditions do varied quasi-experiments reproduce experiments’ results)

·      handling missing data and attrition (when differential T-C attrition is a fierce enemy)

·      the external validity of experimentally-designed evaluations

·      low cost experiments


Consider becoming active in the TIG in order to shape its future and contribute to our field.


Events, Activities & Opportunities

In Winter 2016-17...  The TIG hosted two week’s-worth of postings to the AEA 365 blog (http://aea365.org/blog/). in each of December 2016, we focused on hot topics in experimental evaluation research (see here: http://aea365.org/blog/experiments-tig-week-allan-porowski-on-how-to-make-your-chances-of-conducting-a-successful-rct-seem-a-little-lessrandom/); and in January 2017, we focused on common objections to experiments and why we should not be so concerned (see here: http://aea365.org/blog/experiments-tig-week-the-ethics-of-using-experimental-evaluations-in-the-field-by-laura-peck-and-steve-bell/)


EVAL17... We had a great set of sessions, click here (http://www.evaluationconference.org/p/cm/ld/fid=505), and select the Design & Analysis of Experiments to see the full list.


Annual Business Meeting... Thursday, November 9, 2017, we came together at the conference for "speed meeting," to establish TIG leadership, and to motivate each other to have a great EVAL17 and plan for a great EVAL18!  WE HAVE A NEW WEBMASTER who will be updating this page with profiles of our new co-chairs, program co-chairs, and advisory board members' details, so please check back.


Coming up...  Start thinking now about what you might propose to the 2018 conference.  The more proposals we receive, the better program we can assemble.  Thank you! 

About the Leadership

Laura R. Peck , TIG Co-Chair, is a Principal Scientist at Abt Associates and has 20 years of experience evaluating social welfare and employment policies and programs, both in research and academic settings.  A policy analyst by training, Dr. Peck specializes in innovative ways to estimate program impacts in experimental and quasi-experimental evaluations, and she applies this to many social safety net programs. Prior to joining Abt in 2011, Dr. Peck had been a tenured Associate Professor at the Arizona State University School of Public Affairs, where she taught public policy analysis, program evaluation, and research methods.  At Abt Associates, Dr. Peck is the PI, Co-PI and Director of Analysis for several major national evaluations for the U.S. Departments of Health and Human Services, Labor, Agriculture, and Housing and Urban Development.  Co-author of a public policy text-book, Dr. Peck is well-published on program evaluation topics. Dr. Peck served as Associate Editor (2009-2013) for the American Journal of Evaluation.  She earned her Ph.D. from the Wagner Graduate School at New York University. / Laura_Peck@abtassoc.com




Keith Zvoch, TIG Co-Chair, is an Associate Professor in the Department of Educational Methodology, Policy, and Leadership at the University of Oregon (UO). Dr. Zvoch has over 15 years experience designing and conducting evaluations of educational and social service interventions. At UO, Dr. Zvoch teaches advanced research design and multilevel, multivariate statistics courses. His research interests include the measurement and evaluation of treatment fidelity, the modeling of time series data, and causal inference in applied field settings. Dr. Zvoch is currently an Associate Editor for the American Journal of Evaluation. He has published extensively in education, evaluation, and child development journals. / kzvoch@uoregon.edu

Carl Westine, TIG Co-chair, is an Assistant Professor of Educational Research at University of North Carolina at Charlotte where he teaches graduate level program evaluation and research methods courses.  His research is aimed at finding efficiencies in the planning of evaluations, particularly in the area of STEM education.  He accomplishes this through the empirical estimation of design parameter values needed for power analyses as well as through research on evaluation theories, methods, and practice.  His work has appeared in a variety of evaluation journals including American Journal of Evaluation.  Prior to joining UNC Charlotte, Carl taught at University of West Georgia and also spent several years working in the University of Pittsburgh Medical Center corporate planning office.  Carl is a graduate of the Interdisciplinary Ph.D. in Evaluation Program at Western Michigan University. 

Stephen H. Bell, Program Co-Chair, is a Senior Fellow and Vice President at Abt Associates whose three decades of research focus on measuring the effectiveness of social interventions for disadvantaged Americans.  A specialist in econometric impact analysis, Dr. Bell has helped design and analyze many large-scale random assignment field evaluations.   His current methodological research focuses on ways of making the findings from rigorous impact evaluations more generalizable to the nation and other inference populations, and on “stretching” experimental designs to answer more diverse policy questions and to do so more rapidly.   Dr. Bell’s contributions to the impact evaluation field have been published in numerous peer-reviewed statistics and evaluation journals, a U.S. Department of Labor (DOL) field guide on evaluation methods, and a book on quasi-experimental impact analysis techniques.  He holds a Ph.D. in economics from the University of Wisconsin.

TIG Webmaster: Medha Bulumulla: The responsibilities of the TIG Webmaster include updating and maintaining the content posted on the TIG website, and serving as liason to AEA in regards to the TIG's electronic presence on the AEA website.