A university worked with rural schools to implement a locally-developed integrated science curriculum under a U.S. Math Science Partnership grant. Evaluators found that standard assessments were not sensitive to non-standard interventions. The educators' innovations led to transformation of an evaluation department's skills, and positioned the organization to address new challenges. Lacking relevant science assessments, evaluators and program staff created new evaluation tools to accompany the curriculum. This presentation documents trials, errors, and eventual success as evaluators changed their role and adopted a new suite of assessment skills. Presenters will discuss the logistics of bringing new skills (in this case, Item Response Theory) in-house versus outsourcing. Professional development, time, and technology are all factors in this transition. Presenters will also discuss the implications of these changes for an organization that wants to participate in K-12 educational evaluations in the United States and other countries. #2011Conference #OrganizationalLearningandEvalCapacityBuilding #Prek-12EducationalEvaluation