Statistical methods for examining rater consistency include percent agreement and interrater reliability estimates (e.g., percent agreement, Pearson correlation, Spearman correlation, generalizability coefficient). These documents describe common methods for estimating rater consistency, examine whether the methods result in similar estimates of rater consistency, and present results from a simulation to ascertain which method provides more accurate estimates.#2010Conference #Research,Technology,andDevelopmentEval #AssessmentinHigherEducation #reliability #QuantitativeMethods-TheoryandDesign #interrater_reliability #HowTo