Statistical methods for examining rater consistency include percent agreement and interrater reliability estimates (e.g., percent agreement, Pearson correlation, Spearman correlation, generalizability coefficient). These documents describe common methods for estimating rater consistency,...
Estimating Rater Consistency Presentations.pdf