Panel title: Emerging issues and approaches in environmental program evaluation
Panel abstract: The Environmental Resources Center's Evaluation Unit at the University of Wisconsin will lead this panel on Emerging Issues in Environmental Program Evaluation. An analysis of our Unit's evaluation projects over the last few years revealed several common challenges; panelists will specifically explore challenges related to, 1) measuring medium and long term environmental outcomes, 2) dealing with sensitive, politically charged environmental issues such as climate change, 3) attribution of project outcomes and impacts especially in network and coalition settings, and 4) confronting contentious relationships between stakeholders and clients (for example, between farmers and state environmental agencies).
Panelists will use case studies from their own work and describe the challenges faced, the contexts in which they played out, strategies used to overcome the challenges, and reflect on the success (or failure) of these strategies.
Panelist 1:
Title: Stuck in the middle: Navigating contentious relationships between clients and stakeholders
Abstract: As environmental evaluators working within University and Extension settings, the panelists are often contracted by state, federal, and local environmental agencies to evaluate their programs by involving their stakeholders. Some stakeholders such as farmers, some private landowners, and some community members, have negative perceptions of government agencies, particularly environmental agencies, and are reluctant to be involved with their evaluation efforts. In such cases, evaluators must be especially cognizant when communicating and framing the evaluation. Strategies used by the panelists include collaborating with third-party trusted agencies to be the “face of the evaluation” and playing up their affiliation with the University of Wisconsin-Extension, which is also a trusted agency. Panelists will present these options and discuss whether they were successful or not.
Panelist 2:
Title: Setting reasonable expectations when measuring environmental outcomes
Abstract: In the field of environmental program evaluation, it is not uncommon for clients' logic models to include medium and long term environmental outcomes such as improved water quality or community-level behavior change. The obvious challenges of measuring these outcomes are amplified by short grant periods and insufficient funds for evaluation. In this session, the panelists will use case studies to illustrate these specific challenges and offer some solutions such as developing plans for longitudinal evaluation, offering capacity building sessions to enable clients to perform their own long-term evaluations, and creating templates of instruments that clients can use after the evaluation contract ends.
Panelist 3:
Title: Beyond Increased Knowledge: Encouraging Robust Theories of Change and Evaluation Methods for Environmental Outreach Programs
Abstract:
While the assumption that sharing information will change people’s behavior persists in many sectors, more robust theories of change and approaches to program logic are emerging in the realm of environmental, natural resources and agricultural program evaluation. Evaluators can support the development and application of a suite of theories of change (ie Diffusion of Innovations, the Reasoned Action Approach (RAA, formerly Theory of Planned Behavior), social marketing, social network theory, etc.) that get beyond knowledge transfer. This talk will touch on theories of change utilized in recent environmental and agricultural programs across the Midwestern US. Relevant methods to evaluate a given theory of change will be highlighted, along with some discussion of thinking “beyond surveys.” Surveys are a popular tool because they are familiar, quantitative, convenient, fast etc., but they are not always the only or best approach for measuring program success or failure to achieve intended outcomes.
Relevance statement:
Environmental evaluators face a set of unique challenges related to measuring long term environmental outcomes and dealing with politically charged environmental issues amongst others. This panel consists of three environmental evaluators who have experience dealing with these unique challenges with a variety of government, non-government, and University-based clients. The presentation will draw on their practical experience and will also be informed by scholarly evaluation literature. Environmental evaluators that attend this panel will leave with strategies that they can apply to challenges that they face and will also be able to share their experience during the discussion. The panelists will also discuss their failures and seek feedback from participants on what they could have done differently. We hope that this panel is both a learning and sharing experience for participants and panelists alike.