When evaluations occur over several years, it is sometimes challenging to anticipate how to best measure program impacts. Additionally, as a program evolves based on formative evaluation feedback, it may be difficult to determine what knowledge should be measured at the beginning such that changes in knowledge can be measured and compared later. We served as external evaluators of a federally-funded personnel preparation 5-year grant. Upon graduation of the first cohort, we administered a retrospective pre/post survey of knowledge and skills. Using the same measure, we conducted a pre-assessment of knowledge and skills with cohort 2, followed by a post-assessment at graduation two years later. Comparing results from the retrospective and pre/post assessment of knowledge and skills demonstrated that initial levels and changes in knowledge and skills were very similar, suggesting that retrospective pre/post surveys of knowledge and skills can be useful and valid for measuring program outcomes.