Agreement Coefficients with Missing Ratings
Estimates chance-corrected agreement coefficients (Cohen's,
Conger's, and Fleiss' kappa and the Brennan-Prediger coefficient; Conger
(1980) ) for any number of raters, with
nominal, linear, or quadratic weights, when some ratings are missing.
Missing ratings are handled by inverse-probability weighting (Horvitz and
Thompson (1952) ) when they are
missing completely at random, and by categorical or robust normal-theory
full-information maximum likelihood when they are missing at random
(Little and Rubin (2019) ). Every estimate
comes with an influence-function standard error, giving confidence
intervals and Wald tests that compare coefficients across groups, raters,
or estimators. Also handles counts-format data, multirater g-wise
agreement (Moss (2024) ), and
vector-valued ratings.