Slides from the presidential strand session at AEA 2018
Session title: Are We Part of the Problem or Solution?: Using Systems Approaches to Speak Truth(s) to Power(s) for Better or Worse
Authors: Bob Williams, Miles McNall, & Emily Gates
Session type: Think tank
Abstract: Evaluators wrestle with questions about their roles and responsibilities in addressing social problems and making the world a better place. Questions such as: Should I work on a project whose commissioners espouse different politics and values than mine? How can I negotiate terms of reference that are based on dubious assumptions? To what extent can I critically question the goals of an evaluand? When is it appropriate to examine the broader socio-political systems and issues that influence an evaluand? How can I be inclusive of multiple perspectives amidst power differences? While there are no easy (or agreed on) answers for these dilemmas, systems thinking and approaches offer conceptual and methodological strategies that can help evaluators think thru, navigate, and justify ways of dealing with these dilemmas. This session invites systems-evaluators to share stories, struggles, and successes of using systems approaches to navigate dilemmas of speaking truth(s) to power(s).
Relevance: The theme of “speaking truth to power” calls for a robust discussion on the extent to which evaluators should “stay in their lane.” Many evaluators (and evaluations) focus primarily on the intervention, whether a policy, program, or change initiative, they’re commissioned to evaluate. Bounding evaluations and an evaluator’s role around the intervention, however popular, runs the risk of fixing so-called solutions to problems, rather than addressing the social, political, and economic systems as well as underlying power structures that sustain the problem (e.g., neoliberalism, structural racism and sexism, etc.).
Boundary choices – ranging from defining the purpose(s) of an intervention, distinguishing an intervention from its context, identifying relevant stakeholders, valuing select methods and knowledge forms, and tailoring evaluations to support some (and not other) uses – are ultimately ethical choices. At the heart of these choices lie questions about who we are and what we stand for as evaluators and what role our evaluation profession serves and should serve in society, such as:
- Do evaluators have an ethical responsibility to turn their evaluation armamentarium on social systems that reliably harm the very people served by the programs they are commissioned to evaluate?
- Does evaluation serve as conservative force in society, at most enabling minor changes at the margins while leaving harmful structures in place?
As Ian Davies, ex-President of the European Evaluation Society (EES), said at the EES 2016 conference, are we part of the problem rather than part of the solution? In other words, are we speaking acceptable truths to the powerful rather than transformative truths? Even if we are speaking transformative truths, is anyone listening, acting, or even in positions to act on the ‘truths’ we speak to – especially when calling attention to deep-rooted, systemic inequities?
This think tank will facilitate discussion around the core issue of how systems approaches can help evaluators speak truth(s) to power(s). Systems approaches have grown rapidly and immensely in the evaluation field. Having originated in the systems fields which are defined by a focus on studying and changing complex problems and systems, many systems approaches are designed to help resolve tensions between fundamental issues while coming up with practical results. Some address this by specifically addressing power issues through multiple framings and boundary deliberations, such as soft systems methodology, critical systems heuristics, and feminist systems. Others take a more ‘ontological’ and rational approach such as causal loop diagramming, system dynamics, and social network analysis. This session will seek to explore the range of options that might be open to evaluators wishing to speak truth(s) to power(s). Systems-evaluators, from experienced to novice, will be invited to tell stories about how they’ve used systems approaches to navigate various boundary choices in breakout groups followed by large group discussion about the ethics of speaking truth(s) to power(s).