~AEA Public Library

Back to Library

Eval 14, Session 1250: Development of Tools to Answer Common Questions Regarding Biomedical Research Portfolios 

10-28-2014 13:04

Questions regarding biomedical research portfolios often arise from a variety of audiences, both internal and external. With increasing amounts of administrative, output, and outcome data linked to funded projects, the development of tools to analyze large data aggregations in a standardized manner for analysis and evaluation purposes has become critical. Traditional sources of data on funded projects such as yearly progress reports can be limited in timeliness, scope, and ability to be easily summarized across years, thus not adequately addressing common needs of those seeking to evaluate biomedical research portfolios. The presentations from the National Cancer Institute (NCI), the National Institute of General Medical Sciences (NIGMS), and the National Institutes of Health (NIH) will address the full scope of development of analysis tools for biomedical research portfolios, from initial needs assessment to determine the questions of interest through deployment of enterprise systems available to the public, including lessons learned throughout.

#eval14 #2014Conference

Statistics
0 Favorited
4 Views
5 Files
0 Shares
33 Downloads

Related Entries and Links

No Related Resource entered.

Tags and Keywords

​
Attachment(s)
Download All
pdf file
intro slides   1.23 MB   1 version
Uploaded - 10-28-2014
Questions regarding biomedical research portfolios often arise from a variety of audiences, both internal and external. With increasing amounts of administrative, output, and outcome data linked to funded projects, the development of tools to analyze large data aggregations in a standardized manner for analysis and evaluation purposes has become critical. Traditional sources of data on funded projects such as yearly progress reports can be limited in timeliness, scope, and ability to be easily summarized across years, thus not adequately addressing common needs of those seeking to evaluate biomedical research portfolios. The presentations from the National Cancer Institute (NCI), the National Institute of General Medical Sciences (NIGMS), and the National Institutes of Health (NIH) will address the full scope of development of analysis tools for biomedical research portfolios, from initial needs assessment to determine the questions of interest through deployment of enterprise systems available to the public, including lessons learned throughout.
 
Download
pdf file
NIH acronym guide   82 KB   1 version
Uploaded - 10-28-2014
NIH acronym guide
 
Download
pdf file
Developing an agile tool for monitoring and visualizing ...   1.20 MB   1 version
Uploaded - 10-28-2014
Through a needs assessment using both focus groups and interviews, the National Cancer Institute (NCI) assessed the applicability of an electronic tool to enable greater efficiency in both responding to common information requests and gaining insights into portfolio productivity. In an iterative process with input from key beta testers, the NCI-Viz tool, which aggregates project- and investigator-level information and associated publication metrics, was developed to address the data needs and preferences of the NCI program managers. The tool integrates data from: the NIH project database IMPAC II; PubMed; and the Web of Science®. A user-friendly interface queries funded grants and returns exportable tables of aggregate metrics and practical visualizations. NCI-Viz provides much of the functionality of larger systems but primary development costs are in defining the metrics presented rather than developing a full production system. Preprocessing of data also facilitates tailored alerting systems, a key enhancement to aid grants management.
 
Download
pdf file
Lessons Learned and Future Possibilities   1.50 MB   1 version
Uploaded - 10-28-2014
NIH is a worldwide leader in supporting fundamental biomedical research, which lays foundations for disease diagnosis, treatment and prevention. Evaluating this support requires analysis of diverse investments covering the spectrum of biomedical research, to assess balance, trends, outcomes, and relationships to other agencies. In this presentation we discuss the use of NIHMaps, an online text-mining tool, to analyze research portfolios in NIGMS, an NIH institute whose mission is primarily devoted to basic research. The tool was used for discovery and classification of NIGMS research based on grant scientific titles and abstracts. We describe how this information is complementary to the NIH disease-based research classification (the RCDC system), and how it provided a comprehensive approach for understanding portfolios independent of administrative research programs. We discuss the process for expert validation of the automated categories, and we discuss strengths and weaknesses of the approach and the implications for future text mining efforts.
 
Download
pdf file
Data Analysis Tools and Systems in NIH's Office of Extram...   4.57 MB   1 version
Uploaded - 10-28-2014
In 2013, the NIH created an Office of Data Analysis Tools And Systems (DATA Systems) within its Office of Extramural Research. The DATA Systems mission is to develop databases and tools supporting managers of NIH's $26 billion portfolio of research and training grants, R&D contracts, and intramural research. NIH consolidated multiple ongoing development efforts, creating a central locus within the NIH organization to coordinate development activities and simplify access to a variety of datasets and tools for NIH stakeholders. In this presentation we will describe DATA Systems' activities; demonstrate query and visualization tools that address some of the most common questions faced by research portfolio managers, NIH policy makers, evaluators, and external stakeholders in government and the public and private sectors; and discuss potential new directions NIH could take in providing access to information that fosters government transparency, data-driven policy development, and program evaluation.
 
Download