A Framework for Evaluating Large Scale AIDS Clinical Research Networks
Brendan Cole and Jonathan Kagan, National Institutes of Health
Melissa Burns, Mary Kane, and Kathleen M. Quinlan, Concept Systems Inc.
William Trochim, Cornell University
Daniel Montoya, Hill and Knowlton
Analysis of Follow-up Evaluation Results of Research and Development (R&D) Projects Applying Logic Model to Elucidate the Process of Innovation
Kazuki Ogasahara, Ministry of Economy, Trade and Industry
Osamu Nakamura, National Institute of Advanced Industrial Science and Technology
Kazuyuki Inahashi, Chikahiro Miyokawa, Yoshitaka Kimura, Ministry of Economy, Trade and Industry
Contribution of Evaluation to Management of Research and Development (R&D) in the Process of Technology Transfer: A Knowledge Value Mapping Approach
Juan Rogers, Georgia Institute of Technology
Thursday, November 8
Session: Hard Cases: Measuring and Facilitating Interdisciplinarity and Inter-Organizational Interactions
Chair: Isabelle Collins, Technopolis
University-Industry Collaboration: An issue for Ireland as an Economy With a High Dependence on Academic Research
James Ryan, CIRCA Group Europe Ltd
Measuring the Interdisciplinarity of a Body of Research
David Roessner, SRI International
Anne Heberger, Alex Cohen, Marty Perreault, The National Academies
Alan Porter, Georgia Institute of Technology
Wikis in Evaluation: Evaluating Wikis for Theory Development in a Multi-disciplinary Center
P Craig Boardman, Nathaniel Deshmukh Towery, Brian Zuckerman, Science and Technology Policy Institute
Session: Research Evaluation of the Upcoming Europeans Union’s Framework Programme
Chair: Peter Fisch, European Commission
Networks of Innovation in Information Technology: Technology Development and Deployment in Europe
Nicholas Vonortas, George Washington University
Franco Malerba, Nicoletta Corrocher, Lorenzo Cassi, Luigi Bocconi University
A New System for Research Evaluation Under the European Union's Seventh Framework Programme
Neville Reeve, European Commission
The European Union's Seventh Framework Programme and the Role of Evaluation
Peter Fisch, European Commission
Session: Looking Inside the ResearchCenter Black Box: Using Evaluation Research to Promote Organizational Effectiveness of Scientific Research Centers
Chair: Denis Gray, North Carolina State University
Discussant: Gretchen Jordan, Sandia National Laboratories
Evaluating Leadership Development in an Research and Development (R&D) Context: Assessing Alpha, Beta, and Gamma Change
Bart Craig, North Carolina State University
A Multi-variate Study of Graduate Student Satisfaction and Other Outcomes Within Cooperative Research Centers
Jennifer Schneider, North Carolina State University
Enhancing Collaboration Between Historically Black Colleges and Universities (HBCU) and Research Extensive Universities
Andrea Lloyd, North Carolina State University
Predictors of Cooperative Research Center Post-Graduation Survival and Success
Lindsey McGowen, North Carolina State University
Session: Tools for Useful Performance Assessment of Science and Technology Programs: An Example
Chair: Jerald Hage, University of Maryland
Discussant: Alfred Powell, National Oceanic and Atmospheric Administration
Prologue Jerald Hage
A Strategic Balanced Scorecard for Publicly Funded Science and Technology Programs
Gretchen Jordan, Sandia National Laboratories
Perceptions of the Research Environment: Kinds of Networks, Research, and Projects
Jerald Hage, University of Maryland
Measuring the Immeasurable: Innovation and the Economics Benefits of Satellite Data
Jonathon Mote, University of Maryland
Think Tank:
Identifying Challenges to Using Evaluation to Inform Program Management and Public Policy
Chair: Connie Chang, United States Department of Commerce
Discussants: Connie Chang, United States Department of Commerce and Rosalie Ruegg, TIA Consulting Inc
Business Meeting: Research, Technology, and Development Evaluation TIG Business Meeting
2006 TIG Leaders: Gretchen Jordan, Sandia National Laboratories, George Teather, Independent Consultant, Brian Zuckerman, Science and Technology Policy Institute
Friday, November 9
Session: Evaluation of Community-based Participatory Research and Community Mobilization Strategies to Prevent Chronic Disease and Youth Violence: Advances and Lessons Learned by Two Research Center Programs at the Centers for Disease Control and Prevention
Chair: Alicia Norris, Centers for Disease Control and Prevention
Using Document Review and Data Abstraction to Inform Management of a Federal Research Program: Lessons, Benefits, and Challenges Found by the Centers for Disease Control and Prevention's Prevention Research Centers Program
Demia S Wright, Centers for Disease Control and Prevention
An Evaluation of Community Based Participatory Research and Community Mobilization: Formative Research Results From the National Academic Centers of Excellence on Youth Violence Prevention
Nancy Stroupe, Centers for Disease Control and Prevention
Session: Strategic Evaluation in a Public Research Institute to Contribute to Innovation
Chair: Osamu Nakamura, National Institute of Advanced Industrial Science and Technology
Discussant: Naoto Kobayashi, National Institute of Advanced Industrial Science and Technology
Strategic Evaluation of Research Units Towards Innovation in a Public Research Institute
Osamu Nakamura, Shin Kosaka, Michiko Takagi Sawada, Isao Matsunaga, Masao Koyanagi, Koichi Mizuno, Naoto Kobayashi - National Institute of Advanced Industrial Science and Technology
Evaluation System with PDCA Cycle in the Management of National Institute of Advanced Industrial Science and Technology
Sunao Kunimatsu, Osamu Nakamura, Yoshikazu Arai, Hiroshi Sato, Shinichi Kikuchi, Suzuko Nakatsu, Naoto Kobayashi - National Institute of Advanced Industrial Science and Technology
Session: Peer Review and Learning: New Uses
Chair: David Roessner, SRI International
Peer Review of Transformative Research: Strategies and Challenges in Identifying Innovation in Ex Ante Evaluation
Elmer Yglesias, Science and Technology Policy Institute
David Kaplan, Case Western Reserve University
Peer Reviews or Peers Reviewing? Peer Review as Policy Learning in Innovation, Research and Education
Erik Arnold, Isabelle Collins, Technopolis, UK
Session: Ex Ante Evaluation: Methods for Estimating Innovation and Other Research Outcomes
Chair: George Teather, Independent Consultant
Ex Ante Portfolio Analysis of Public R&D Programs for Industrial Technologies in Korea: Practices at the Korea Institute of Industrial Technology Evaluation and Planning
Yongsuk Jang, George Washington University
Jongman Park, Korea Institute of Industrial Technology Evaluation and Planning
Impact Evaluation in Preliminary Feasibility Analysis of National R&D Programs
Jiyoung Park, Korea Institute of Industrial Technology Evaluation and Planning
Session: A Directory of Evaluation Methods for Managers of Public Research and Technology Programs
Rosalie Ruegg, TIA Consulting Inc
Saturday, November 10
Session: Ex Ante Evaluation: Evaluation as an Agent of Program Change: An Example From Austria
Chair: Klaus Zinoecker, Vienna Science and Technology Fund
The Evaluation of Genome Research Austria (GEN-AU): Overview of the Study's Aims, Structure, Methods, Results, Implications, and Impacts
Klaus Zinoecker, Vienna Science and Technology Fund, Alfred Radauer, Iris Fischl, Roald Steiner - Austrian Institute for SME Research, Brigitte Tempelmaier, Austrian Economic Service, Rosalie Ruegg, TIA Consulting Inc
Developing a Plan for Future Monitoring and Impact Analysis of Genome research Austria
Rosalie Ruegg, TIA Consulting Inc
Session: A Roadmap for Developing a Public Health Research Portfolio Evaluation Program
Chair: Robin Wagner, Centers for Disease Control and Prevention
Overview of Methodology Used to Develop a Research Evaluation Program
Jerald O'Hara, John Araujo, Mona Choi, Catherine Pepper, Robin Wagner, Guijing Wang, Trevor Woollery, Centers for Disease Control and Prevention
MEDLINE Search Strategies vs. Relevant Retrieval: How Closely do They Match for a Research Evaluation Topic?
Catherine Pepper, Jerald O'Hara, John Araujo, Mona Choi, Robin Wagner, Guijing Wang, Trevor Woollery, Centers for Disease Control and Prevention
Evaluating Public Research Investment: A Literature Review
Jerald O'Hara, John Araujo, Mona Choi, Catherine Pepper, Robin Wagner, Guijing Wang, Trevor Woollery, Centers for Disease Control and Prevention
Extending the Pay-Back Model to Incorporate Costs as Well as Benefits to Measure the Net Impacts of Organizational Expenditures on Public Health Research
Guijing Wang, Jerald O'Hara, John Araujo, Mona Choi, Catherine Pepper, Robin Wagner, Trevor Woollery, Centers for Disease Control and Prevention
A Bibliometric Methodology to Inform a Logic Model for Evaluating a Public Health Research Portfolio
John Araujo, Jerald O'Hara, Mona Choi, Catherine Pepper, Robin Wagner, Guijing Wang, Trevor Woollery, Centers for Disease Control and Prevention
Session: Using Logic Models to Evaluate Research and Technology Diffusion Results: Two Cases
Chair: Jeff Dowd, United States Department of Energy
Cutting Edge Logic Models for Research and Technology Programs
Gretchen Jordan, Sandia National Laboratories
Linking Projects to Program Outcomes in Metrics for Technology Development Programs
John Mortensen, Energetics, Inc.
The Logic of Indirect Programs to Diffuse Technologies: The Example of Training
John Reed, Innovologie
Getting From Training to Credible Energy Savings: An Evaluation Template
Harley Barnes, LM Business Process Solutions
Session: The Follow-up Monitoring and Outcome Survey for National Research and Development Projects in New Energy and Industrial Technology Development Organization (NEDO)
Chair: Takahisa Yano, New Energy and Industrial Technology Development Organization
Study of the Correlation Between Ex-post Evaluation and Follow-up Monitoring of National Research and Development (Part I)
Hiroyuki Usuda and Momoko Okada, New Energy and Industrial Technology Development Organization
Study for the Important Management Factors Based on Follow-up Monitoring Data (Part II)
Setsuko Wakabayashi, Tsutomu Kitagawa, Takahisa Yano, Kazuaki Komoto, New Energy and Industrial Technology Development Organization
Approach for the Understanding of Outcomes Derived from National Research and Development of Energy Conservation Project (Part III)
Kazuaki Komoto, Tsutomu Kitagawa, Takahisa Yano, Setsuko Wakabayashi, New Energy and Industrial Technology Development Organization
Session: Forging a Strong Link Between Research and Science Policy for Air Quality Decisions
Chair: Dale Pahl, United States Environmental Protection Agency
An Overview of National Ambient Air Quality Standards
Ron Evans, United States Environmental Protection Agency
A Paradigm for Federal Particulate Matter Research
James Vickery, United States Environmental Protection Agency
Relationships Among Atmospheric Contaminants, Air Quality, Human Exposure, and Health
Rochelle Araujo, United States Environmental Protection Agency
Synthesis and Evaluation of New Scientific Knowledge
William Wilson, United States Environmental Protection Agency
Session: National Performance Evaluation System of Research and Development Programs in Korea: System and Applications
Chair: Sang-Youb Lee, Korea Institute of Science and Technology Evaluation and Planning
Discussant: Jiyoung Park, Korea Institute of Science and Technology Evaluation and Planning
Method and Application of the Survey and Analysis of National Research and Development (R&D) Programs for the Performance Evaluation in Korea
Keun-Ha Chung, Hyejung Joo, Herin Ahn, Korea Institute of Science and Technology Evaluation and Planning
The Performance Evaluation of Research and Development Programs in Korea
Seong-jin Kim, Soon Cheon Byeon, Korea Institute of Science and Technology Evaluation and Planning
Design of Metaevaluation Model for National Research and Development Programs in Korea
Young Soo Ryu and Soon Cheon Byeon, Korea Institute of Science and Technology Evaluation and Planning
Byung Dae Choi, Hanyang University
Development of the Evaluation Methodology for the Basic Research in Korea
Hyeyoung Yang and Sangki Jeong, Korea Institute of Science and Technology Evaluation and Planning
Performance Evaluation of Agriculture Research and Development Programs in Korea
Hoijong Jung, Korea Institute of Science and Technology Evaluation and Planning
Session: Centralized E-Tool for Organizational Performance Management: National Institutes of Health (NIH) Government Performance Result Acts (GPRA) & Program Assessment Rating Tool (PART) Assessments
Deborah Duran, National Institutes of Health