Evaluation 2020

2453: Where Has all the Data Gone? A Tool that Can Help Answer that Question. 

10-26-2020 09:06

Foundations invest a lot of time and money into gathering data as part of evaluations they fund but then that data is often not available to others who could benefit from it. This paper discusses how support for data archiving – specifically using a tool called the Data Value Map - can expand evaluation use and shine a light on the original data gathered. However, ensuring data can be archived when evaluations are concluded can meet with obstacles such as identifying files, anonymisation and consent to name a few. Using this new visual tool as a lens to explore this further it was apparent that many of the barriers occurred when the data was created. Knowing at what stage of the data management process there is a bottleneck is one important step to help ensure data is accessible, ready to share and potentially contribute to future analysis in ways not considered initially.

Preserving and sharing evaluation data can help extend the influence of evaluations and prolong the life of the data. This poster aims to demonstrate how preserving a rich body of high-quality evaluation data, that has already made a unique contribution to evidence-based policy making, can inform further analysis and new studies as well as add to the knowledge base. An analysis of the issues encountered in retrospectively preparing data for archiving, using a visual tool called the Data Value Map (Nagle and Sammon, 2017), sheds light on issues to be avoided and helps to identify practices that would ensure data is “archive-ready” from the design stage of a grant. The visualisation of the process, afforded by the Data Value Map, may also help researchers better understand all the components of data management and prompt consideration by the data creator of the wider life of research data.

The paper is informed by a $200 million programme funded by The Atlantic Philanthropies (2004-2013) with the goal of transforming the way that children and young people receive services on the island of Ireland. An unprecedented programme, it delivered 52 evidence-based services to children, young people and their families. Grantees were also funded to evaluate the services using experimental designs to the extent that was possible. This was an innovation in its own right as such studies had rarely been done to assess children's services. Extensive and wide-ranging evaluation datasets generated an exceptional evidence base, informing the work of practitioners and policy makers and securing additional government support to continue evidence-informed interventions. There was much to be gained from further exploration of the findings and a learning initiative, managed by the Children’s Research Network for Ireland and Northern Ireland (CRNINI), was supported for archiving and further analysis of the data to embed the legacy and learning. It also provided a unique insight into the work required and the challenges encountered in the creation a data bank to archive project evaluation datasets.

Data is an important output of philanthropic investment and this study demonstrates how it’s possible for foundations and non-profit organisations to leverage their data to facilitate additional analysis, link to other datasets and encourage broader learning in their fields. This is increasingly relevant as many national and international research funders require original data to be made available to others alongside academic publications to maximise the investment and further exploration on the data.

This poster may also be of interest to foundations and non-profit organisations interested in the power of data, informing how they might collect their data and make it available to others to help solve social problems. It may also encourage foundations to consider data archiving as part of a broader commitment to knowledge sharing and secondary analysis and to ramp up their data capacity and that of the non-profit organisations they support.

Birkbeck, G. (2018) Archiving data from the Prevention and Early Intervention Initiative: A funders perspective. Children’s Research Digest, 4 (3). Available at: https://childrensresearchnetwork.org/knowledge/resources/archiving-data-from-the-prevention-and-early-intervention-initiative-a-funders-perspective

Nagle, T. and Sammon, D. (2017). "THE DATA VALUE MAP: A FRAMEWORK FOR DEVELOPING SHARED UNDERSTANDING ON DATA INITIATIVES". Available at: http://aisel.aisnet.org/ecis2017_rp/93

0 Favorited
1 Files

Related Entries and Links

No Related Resource entered.

Tags and Keywords

pdf file
Poster   745 KB   1 version
Uploaded - 10-26-2020
Data management planning