Measurement of interobserver disagreement: correction of Cohen's kappa for negative values
type: Item
instance of: scholarly article
title: Measurement of interobserver disagreement: correction of Cohen's kappa for negative values
zbMATH Open document ID: 1426.62175
DOI: 10.1155/2015/751803
author: Tarald O. Kvålseth
published in: Journal of Probability and Statistics
publication date: 2018-08-14T00:00:00Z
review text: Summary: As measures of interobserver agreement for both nominal and ordinal categories, Cohen's kappa coefficients appear to be the most widely used with simple and meaningful interpretations. However, for negative coefficient values when (the probability of) observed disagreement exceeds chance-expected disagreement, no fixed lower bounds exist for the kappa coefficients and their interpretations are no longer meaningful and may be entirely misleading. In this paper, alternative measures of disagreement (or negative agreement) are proposed as simple corrections or modifications of Cohen's kappa coefficients. The new coefficients have a fixed lower bound of \(-1\) that can be attained irrespective of the marginal distributions. A coefficient is formulated for the case when the classification categories are nominal and a weighted coefficient is proposed for ordinal categories. Besides coefficients for the overall disagreement across categories, disagreement coefficients for individual categories are presented. Statistical inference procedures are developed and numerical examples are provided.
Mathematics Subject Classification ID: 62H20
zbMATH DE Number: 6917434
Wikidata QID: Q59114241
cites work: bootlib
cites work: Statistical Methods for Rates and Proportions
cites work: Q5390297
cites work: New Interpretations of Cohen’s Kappa
cites work: Q3150769
cites work: Q4370586
cites work: Jackknifing in Categorical Data Analysis
describes a project that uses: bootlib
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.1155/2015/751803
OpenAlex ID: W1951680662
description: scientific article; zbMATH DE number 6917434