The Program Evaluation Standards have been widely accepted as the prevailing criteria for assessing evaluation quality in North America. They were designed to be applicable to a broad array of evaluation contexts. Their generality makes them adaptable for different settings and uses, but also leaves them open to substantial interpretation by users. Although the Standards were not put forth as a rating tool, they are commonly used in that capacity for metaevaluation purposes. Problems related to consistency in the application of the Standards are exacerbated when information about the evaluation(s) being assessed is limited to what is documented in evaluation reports, since many standards refer to aspects of evaluations that are not commonly detailed in writing. Based largely on a study that investigated interrater reliability in metaevaluation, this presentation describes the pitfalls associated with applying the standards to written reports for metaevaluation purposes and identifies ways to improve consistency in metaevaluation. #2010Conference #metaevaluation #standards #ResearchonEvaluation