Community References

Evaluating transformative research programmes: A case study of the NSF Small Grants for Exploratory Research programme. Caroline S. Wagner and Jeffrey Alexander. Research Evaluation 22 (2013) pp. 187–197

Excellent, addresses looking across fields, has chapter on assessment and power

National Research Council (US) 2007.  A Strategy for Assessing Science: Behavioral and Social Research on Aging. Edited by Irwin Feller and Paul C Stern. Committee on Assessing Behavioral and Social Science Research on Aging. Washington (DC): National Academies Press (US).


Tackles measuring efficiency of research

National Research Council 2008. Evaluating Research Efficiency in the U.S. Environmental Protection Agency. Washington, DC: The National Academies Press, 2008.


A comprehensive framework from advancing knowledge and capacity to social and economic outcome

Canadian Academy of Health Sciences, Panel on Return on Investment in Health Research. (2009). Making an impact: A preferred framework and indicators to measure returns on investment in health researchOttawa (ON), Canada: Canadian Academy of Health Sciences (CAHS). 


Work done for DOE Office of Science

Feller, Irwin, George Gamota, and William Valdez. "Developing science indicators for basic science offices within mission agencies." Research Evaluation12.1 (2003): 71-79.


Recent, all about R&D  even though book title does not indicate that

Link AN, & Vonortas NS. (2013). Handbook on the theory and practice of program evaluation. Cheltenham: Edward Elgar


Excellent, explains Contribution Analysis as complement/alternative to economic impact studies

Mayne J. and Stern E.. Impact evaluation of natural resource management research programs: a broader view. ACIAR Impact Assessment Series Report. 2013,, last accessed 1 July, 2014.


Very good framework, relies mostly on expert review

National Academies Committee for the Review of NIOSH Research Programs. (2007).Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health. National Research Council;

RAND. (2009).Demonstrating and Communicating Research Impact:  Preparing NIOSH Programs for External Review. Santa Monica, CA: RAND;


Comprehensive, conveys all that was learned from evaluating that program

Ruegg, Rosalie, and Irwin Feller. A Toolkit for Evaluating Public R & D Investment: Models, Methods, and Findings from ATP's First Decade. US Department of Commerce, Technology Administration, National Institute of Standards and Technology, 2003.


A few pages on each method, with pros/cons, examples

Ruegg, Rosalie, and Gretchen Jordan, 2007.  Overview of Evaluation Methods for R&D Programs, U.S. DOE, March.


Recent literature reviews on evaluation of big science facilities

Autio, Erkko, Imperial College London. UK Department for Business Innovation and Skills, Innovations from Big Science: Enhancing Big Science Impact Agenda. March 2014.

Technopolis Group, Big Science and Innovation, July 2013.

Technopolis Group, “The role and added value of large-scale research facilities”, final report, Brighton, UK, 10 February, 2011