Evaluation capacity building (ECB) focuses on facilitating the development of individual and organizational competencies and structures—such as evaluation knowledge and an organizational culture of evaluation—that promote sustained, high quality evaluation. Evaluative thinking is also mentioned in the ECB literature as an important attribute, yet such references are often fleeting. In this paper, we first present our rationale for highlighting evaluative thinking as an important component of ECB practice and as an object of inquiry within research on evaluation. Second, we draw on cognitive and education research to help develop and clarify the construct of “evaluative thinking.” Finally, we explore some ways of operationalizing and measuring this construct, considering both qualitative and quantitative methods. Our exploratory work on measuring evaluative thinking is situated in a project designed to promote evaluative thinking and foster high quality internal evaluation among non-formal science, technology, engineering and math (STEM) educators. #EvaluativeThinking