AEA 2013


Wednesday, October 16

Session 8: Established and Emerging Evaluation Foci: Revisiting Technology Assessments and Exploring Blogs and Automation
Chair: 
Laurel Haak,  ORCID
 
Methods for Designing Technology Assessments
Virginia Chanley and Umesh Thakkar, United States Government Accountability Office

Evaluating the Diffusion of Information From a Federal Research Agency Blog: A Case Study From the National Institutes of Health (NIH)
Katherine Catevenis, Lindsey Sam Ryan, Nicole Garbarini, David Rosen, Michael Dorsey, Katrina Pearson, Megan Columbus, and Robin Wagner, National Institutes of Health

A New Approach for Automated Author Discipline Categorization and Evaluation of Cross-Disciplinary Collaborations for Grant Programs
Ilya Ponomarev, Pawel Sulima, Unni Jensen, and Joshua Schnell, Thomson Reuters
Jodi Basner, Discovery Logic, a Thomson Reuters Business
Karen Jo and Nicole Moore, National Institutes of Health

Session 33: Evaluation of Research Process and Impact: A Continuum of Methods Utilized in Evaluating the Clinical and Translational Science Awards
Chair:  Adrienne Zell, Oregon Health and Sciences University

Identification of Translational Research Strengths, Opportunities, and Barriers Within Academic Medical Centers: A Mixed Methods Approach
Adrienne Zell, Lindsey Smith, and David Dilts, Oregon Health and Sciences University

Documenting Complex Workflows and Providing Recommendations for Streamlining: An Analysis of Regulatory Approval Processes at the Colorado Clinical and Translational Sciences Institute
Jeff Proctor and Kathryn Nearing, University of Colorado at Denver

Evaluation Approaches and the Promotion of an Evaluative Culture: Lessons Learned in a CTSA Context
Barbara Thomas, University of Michigan

Promoting Translational Science through Pilot Projects: Using a Mixed-Method Return on Investment (ROI) Approach with 400 Studies Funded from 2008-2012 through NC TraCS (UNC Clinical and Translational Science Award)
Rita O'Sullivan, Mary Beth Lister, and Jessie Montana Cain, University of North Carolina at Chapel Hill

The Need for Training on Team and Meeting Management to Enhance Capacity for Team Science: Converging Lines of Evidence From a Comprehensive Evaluation of the Colorado Clinical and Translational Sciences Institute
Kathryn Nearing and Will Paterson, University of Colorado at Denver

Thursday, October 17

Session 111: New Perspectives on Research and Development Evaluation Systems for Performance Goal-centered Evaluation in Korea Chair:  Sang Youb Lee, Korea Institute of S&T Evaluation and Planning (KISTEP)

The Performance Management and Evaluation System of National Research and Development (R&D) Programs in Korea 
Ilhwan Lee and Su Youn Kim, Korea Institute of S&T Evaluation and Planning (KISTEP)
Junghoe Bae and Bongrak Lee, Ministry of Science, ICT and Future Planning (MSIP)

New Measures for Evaluation System of Government-funded Research Institutes (GRIs) in Korea
Woo Chul Chai and Mee Kim, Korea Institute of S&T Evaluation and Planning (KISTEP)

A New Approach for Strengthening Research and Development (R&D) Capacity in SMEs: A Case Study on National R&D Programs in Korea
Jae-Ho Shin, Ilhwan Lee, and Munsang Kang, Korea Institute of S&T Evaluation and Planning (KISTEP)

In-depth Evaluation of Horizontal Science and Technology (S&T) Labor Force Programs in Korea: A Case Study of Active Labor Market Policies (ALMP) Related Programs
Heekweon Lee, Mee Kim, and Ji-Eun Lim, Korea Institute of S&T Evaluation and Planning (KISTEP)

Session 151: Impact of Farmer Research Group Interventions on Maize Farmers in Central Rift Valley of Oromia: An Empirical Study
Presenter:  Wole kinati Wakjira, Chinese Academy of Sciences

Session 166: Promoting Regional Research and Development (R&D) Innovation through Evaluation: Perspectives From Europe and Argentina
Chair:  Juan Rogers,  Georgia Institute of Technology

Evaluating the Use of Peer Review in the Development of Regional Research and Innovation Strategies for Smart Specialisation
Ruslan Rakhmatullin, European Commission

Designing an Integrated System of Evaluation to Promote Innovation With a Territorial Approach: The Case of INTA Argentina
Eliseo Monti, Susana Mirassou, Hernán Trebino, Patricia Arnozis, and José Portillo, Instituto Nacional de Tecnología Agropecuaria

Session 221: Opening the Black Box of Innovation: Interim Impacts Early in a Product Life Cycle
Chair:  Gretchen Jordan, 360 Innovation LLC

The Logic of Public Interventions to Accelerate Development of New Products and Supply Chains
Gretchen Jordan, 360 Innovation LLC

A Four-tiered, Question-driven Evaluation Approach to Measure Interim Progress
Rosalie Ruegg, TIA Consulting Inc.

Exploring the Use of Netchain Analysis in Evaluating Emerging Markets
Jonathan Mote, George Washington University

Session 300: Theory and Methods to Assess Collaborations and Interdisciplinary Research
Chair: Robin Wagner,  National Institutes of Health

Evolution of Virtual Research Collaborations
Arsev Aydinoglu, NASA Astrobiology Institute
Chad Mitchell and Suzie Allard, University of Tennessee at Knoxville

A Model for Collaboration in Clinical Research Studies
Richard Fabsitz, Jean Olson, Mona Puggal, Kara Hall, Matthew Eblen, and Robin Wagner, National Institutes of Health

Session 351: Testing a Strategic Evaluation Framework for Incrementally Building Evaluation Capacity in a Federal Research and Development Program
Chair(s):
Michael Coplen, Rick Kowalewski, Joe Carra, and John Tunna, United States Department of Transportation
Bill Valdez, United States Department of Energy
Stephen Godwin, The National Academies

Friday, October 18

Session 426: Strategic Research Program Evaluation to Realize Outcomes Toward Impacts for Innovation in Japan
Chair:  Naoto Kobayashi, Waseda University
Discussant:  Osamu Nakamura, National Institute of Advanced Industrial Science and Technology (AIST) Chugoku

An Improved Approach of Evaluation of Research and Development in AIST
Masaharu Takahashi, Chieko Kifune, Hitoshi Akimichi, Tooru Ikgami, Hidenori Endo, and Kanji Ueda, National Institute of Advanced Industrial Science and Technology (AIST)

Strategic Evaluation of Regional Bases of AIST (Advanced Industrial Science and Technology) to Meet the Impacts for Innovation
Osamu Nakamura and Shinichi Matsui, National Institute of Advanced Industrial Science and Technology (AIST) Chugoku
Naoto Kobayashi, Waseda University

Analysis of 'NEDO Inside Products' Survey 2013
Yoshiko Yurugi, Masaru Yamashita, Sayaka Shishido, Tomonaga Yoshida, and Mituru Takeshita, NEDO

Trial of Numerical Analysis for NEDO (New Energy and Industrial Technology Development Organization) Projects Follow up Monitoring
Sayaka Shishido, Masaru Yamashita, Tomonaga Yoshida, Yoshiko Yurugi, and Mituru Takeshita, NEDO

Session 481: Analytic Methods Applied to Research Project Publication Data to Assess Either Collaboration or Productivity
Chair:  James Corrigan, National Institutes of Health

Multivariate Analysis of Collaboration Patterns Among Researchers in Two Epidemiological Cohort Studies
Matthew Eblen, Katherine Catevenis, Richard Fabsitz, Jean Olson, Katrina Pearson, Mona Puggal, Deepshikha RoyChowdhury, and Robin Wagner, National Institutes of Health

Data Modeling to Support Portfolio Analysis and Productivity Assessment
Duane Williams, Leo DiJoseph, and Joshua Schnell, Thomson Reuters
James Corrigan, Danielle Daee, Elizabeth Hsu, and Lawrence Solomon, National Institutes of Health

Session 586: Evaluating Biomedical Research Training Programs: Examples From the National Institutes of Health
Chair:  Julie Mason, National Institutes of Health
Discussant(s):
Robin Wagner, National Institutes of Health
Jack Mills, Independent Consultant

Best Practices: Leveraging Existing Data Resources to Evaluate National Institutes of Health (NIH) Research Training Programs
Jennifer Sutton, Deepshikha Roychowdhury, Cassandra Spears, Katrina Pearson, and Robin Wagner, National Institutes of Health

Career Outcomes Evaluation of Alumni in a Structured Postdoctoral Training Program
Jessica Faupel-Badger, National Institutes of Health

Where Are They Now? Tracking Long Term Trainee Outcomes at the National Institutes of Health
Helena Kennedy, National Institutes of Health

Clinical Research/Reproductive Scientist Training (CREST): Qualitative Evaluation of a Research Training Program
Sarah Glavin and Jamelle Banks, National Institutes of Health
Mary Dufour, Nekisha Lakins, Margaret Blasinsky, and Jack E. Scott, The Madrillon Group Inc.

Session 612: Assessing the Contributions of Research Capacity and Innovation to Outcomes and Impacts
Chair:  Brian Zuckerman, Science and Technology Policy Institute
Discussant:  Kathryn Graham, Alberta Innovates Health Solutions

Research and Practice of Quantitative Monitoring System of Chinese Academy of Sciences
Haijun Zheng, Zhongcheng Guan, Biaoxiang Wang, and Jianmei Wu, Chinese Academy of Sciences

Session 636: Evaluating Research Portfolios I: An Analytical Perspective
Chair:  Robin Wagner, National Institutes of Health
Discussant:  Nicholas Vonortas, George Washington University

Combining Evaluation Methods for Assessment of Large Research and Development (R&D) Portfolios in Energy Efficiency and Renewable Energy
Rosalie Ruegg, TIA Consulting Inc.

R&D Portfolio Valuation: Asymmetric Project Payoffs, Project Relationships, and the Question of Whether a Portfolio is More Than an Aggregation of Projects
Peter Linquiti, George Washington University

Connections: Research Knowledge Pattern Assessment
Alan Porter, Georgia Institute of Technology

Worth more than you think: Non Gaussian Behavior of R&D Projects and its Implications to Portfolio Management
Jonathan Linton, University of Ottawa
Sebastien Casault, University of Twente

Session 691: Evaluating Research Portfolios II: A Practitioner's Perspective
Chair:  Nicholas Vonortas, George Washington University
Discussant:  Jonathan Linton, University of Ottawa

A Policymaker's Perspective on Portfolio Management of Federal Research and Development
Kei Koizumi, Office of Science and Technology Policy

Science of Science Policy & Portfolio Evaluation
Bill Valdez, United States Department of Energy

Portfolio Analysis at the National Institutes of Health
George Santangelo, National Institutes of Health

Portfolio Analysis at the International Development Research Centre (IDRC)
Colleen Duggan and Amy Etherington, International Development Research Centre

Saturday, October 19

Session 746:  A Methodology to Develop Logic Maps for Basic Research Funding Programs
Shannon Griswold, Alexandra Medina-Borja, and Konstantinos Triantis, National Science Foundation

Kathryn Graham and Heidi Chorzempa, Alberta Innovates Health Solutions

Session 977: New Data and New Approaches for Assessment of Emerging Technologies: World Wide Web, Social Media and "In-Depth" Bibliometrics
Chair:  Juan Rogers, Georgia Institute of Technology

Jan Youtie and Sanjay Arora, Georgia Institute of Technology
Philip Shapira, Manchester University

Sanjay Arora, Georgia Institute of Technology

Alan Porter, Georgia Institute of Technology 
Yi Zhang and Xiao Zhuo, Beijing Institute of Technology

Juan Rogers, Georgia Institute of Technology