~AEA Public Library

​The objective of this handout is to share lessons learned from an evaluation process for a new technology to help inform other AEA evaluators working to assess and improve professionals’ performance.
We discuss the background, research questions, study design, outcomes, original and revised data collection tool and process based on the challenged encountered, the efforts at data quality along with the pros and cons, planned analyses. The three lessons learned are:

• Data from a review of a record of technology should be available at the granular level in the electronic form for further data analysis.
• From the beginning, it is good to anticipate the building of an evaluation database into the back-end of the software app decision-support tool for providers.
• Data quality and ensuring accuracy (validity) and consistency (reliability) of data is a process that can be planned for in evaluations.