The Measurement of Observer Agreement for Categorical Data

DOI10.2307/2529310zbMath0351.62039OpenAlexW2164777277WikidataQ26778373 ScholiaQ26778373MaRDI QIDQ4121322

Publication date: 1977

Published in: Biometrics (Search for Journal in Brave)

Full work available at URL: https://semanticscholar.org/paper/7e7343a5608fff1c68c5259db0c77b9193f1546d

Mathematics Subject Classification ID

Applications of statistics to biology and medical sciences; meta analysis (62P10) Hypothesis testing in multivariate analysis (62H15)

Related Items (88)

The effect of combining categories on Bennett, Alpert and Goldstein's \(S\) ⋮ Equivalences of weighted kappas for multiple raters ⋮ Conditional inequalities between Cohen's kappa and weighted kappas ⋮ A statistical test for detecting discordance in rankings between k groups ⋮ Weighted kappa statistic for clustered matched-pair ordinal data ⋮ AGREEMENT ON A TWO–POINT SCALE ⋮ Kappa coefficients for circular classifications ⋮ Sample size requirements for interval estimation of the intraclass kappa statistic ⋮ Partially identified prevalence estimation under misclassification using the kappa coefficient ⋮ Multistage Evaluation of Measurement Error in a Reliability Study ⋮ Mathematical modelling of the mosquito \textit{Aedes polynesiensis} in a heterogeneous environment ⋮ Sample size determinations for the two rater kappa statistic ⋮ Segmentation and classification of high resolution imagery for mapping individual species in a closed canopy, deciduous forest ⋮ A sensibility study of the autobinomial model estimation methods based on a feature similarity index ⋮ Assessing interrater agreement on binary measurements via intraclass odds ratio ⋮ Statistical inference of agreement coefficient between two raters with binary outcomes ⋮ Statistical inference of Gwet’s AC₁ coefficient for multiple raters and binary outcomes ⋮ Kappa statistic for clustered physician-patients polytomous data ⋮ Weighted kappas for \(3 \times 3\) tables ⋮ Confidence intervals on intraclass correlation coefficients in a balanced two-factor random design ⋮ Weighted inter-rater agreement measures for ordinal outcomes ⋮ Collective intertemporal decisions and heterogeneity in groups ⋮ Statistical evaluation of a fully automated mammographic breast density algorithm ⋮ Boolean function metrics can assist modelers to check and choose logical rules ⋮ Ramifications of a population model for \(\kappa\) as a coefficient of reliability ⋮ Content validation template (CVT) ⋮ Latent trait models for perceived risk assessment using a Covid-19 data survey ⋮ Musical genres: beating to the rhythms of different drums ⋮ Mathematical regularities of data from the property listing task ⋮ Identifying and supporting academically low-performing schools in a developing country: an application of a specialized multilevel IRT model to PISA-D assessment data ⋮ Improving precision through modelling: an illustration with hierarchical kappa ⋮ The dependence of chance-corrected weighted agreement coefficients on the power parameter of the weighting scheme: analysis and measurement ⋮ Using unfolding analysis and MARS approach for generating a scoring system from a group preference information ⋮ Dengue transmission under future climate and human population changes in mainland China ⋮ Cohen's weighted kappa with additive weights ⋮ Conditional symmetry models for three-way contingency tables ⋮ Forecasting jump arrivals in stock prices: new attention-based network architecture using limit order book data ⋮ On the equivalence of multirater kappas based on 2-agreement and 3-agreement with binary scores ⋮ Analysis of robust design experiments with time-dependent ordinal response characteristics: a quality improvement study from the horticulture industry ⋮ Modeling sensitivity and specificity with a time-varying reference standard within a longitudinal setting ⋮ The use of variance components for the assessment of outcome measures in rheumatology ⋮ Testing stochastic orders in tails of contingency tables ⋮ Inequalities between multi-rater kappas ⋮ Misreported schooling, multiple measures and returns to educational qualifications ⋮ Coefficient of variation: the second-order alternative ⋮ A unified treatment of agreement coefficients and their asymptotic results: the formula of the weighted mean of weighted ratios ⋮ On Modelling Agreement and Category Distinguishability on an Ordinal Scale ⋮ Structural analysis of subjective categorical data ⋮ KNN and adaptive comfort applied in decision making for HVAC systems ⋮ New Agreement Measures Based on Survival Processes ⋮ Inferences on the intraclass correlation coefficients in the unbalanced two-way random effects model with interaction ⋮ Sklar's Omega: A Gaussian Copula-Based Framework for Assessing Agreement ⋮ Beyond kappa: A review of interrater agreement measures ⋮ Dispersion-weighted kappa: an integrative framework for metric and nominal scale agreement coefficients ⋮ Statistical description of interrater variability in ordinal ratings ⋮ Chance-corrected measures of reliability and validity in K K tables ⋮ Assessing the reliability of ordered categorical scales using kappa-type statistics ⋮ Nonparametric Estimation of the Concordance Correlation Coefficient under Univariate Censoring ⋮ Corrected asymptotic distribution of statistics based on the multinomial law ⋮ Multi-state deterioration prediction for infrastructure asset: learning from uncertain data, knowledge and similar groups ⋮ Estimation of symmetric disagreement using a uniform association model for ordinal agreement data ⋮ Kappa coefficients for dichotomous-nominal classifications ⋮ Recurring concept memory management in data streams: exploiting data stream concept evolution to improve performance and transparency ⋮ Empirical study for the agreement between statistical methods in quality assessment and control of microarray data ⋮ Equivalence of kernel machine regression and kernel distance covariance for multidimensional phenotype association studies ⋮ Who inflates the bubble? Forecasters and traders in experimental asset markets ⋮ Measuring Agreement of Multivariate Discrete Survival Times Using a Modified Weighted Kappa Coefficient ⋮ Inference Procedures for Assessing Interobserver Agreement among Multiple Raters ⋮ Weighted Least-Squares Approach for Comparing Correlated Kappa ⋮ Inconsistencies in reported employment characteristics among employed stayers ⋮ Estimating multiple rater agreement for a rare diagnosis ⋮ Average kappa coefficient: a new measure to assess a binary test considering the losses associated with an erroneous classification ⋮ Ensemble of naïve Bayesian approaches for the study of biofilm development in drinking water distribution systems ⋮ Modeling Association plus Agreement among Multi-Raters for Ordered Categories ⋮ A Statistical Approach to Assessment of Agreement Involving Multiple Raters ⋮ Fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data: a perspective of generalized inverse ⋮ Confidence Intervals of Weighted Kappa Coefficient of a Binary Diagnostic Test ⋮ A comparison of reliability coefficients for ordinal rating scales ⋮ Test-inversion confidence intervals for estimands in contingency tables subject to equality constraints ⋮ Interplay of rumor propagation and clarification on social media during crisis events -- a game-theoretic approach ⋮ How to assess rater rankings? A theoretical and a simulation approach using the sum of the pairwise absolute row differences (PARDs) ⋮ A general method of estimating measures of association and their asymptotic variances under the multinomial model using standard statistical software ⋮ Accountability as a warrant for trust: an experiment on sanctions and justifications in a trust game ⋮ On population‐based measures of agreement for binary classifications ⋮ Unnamed Item ⋮ Probability weighting and utility curvature in QALY-based decision making ⋮ A two-step machine learning approach to predict S&P 500 bubbles ⋮ Methods of assessing categorical agreement between correlated screening tests in clinical studies

This page was built for publication: The Measurement of Observer Agreement for Categorical Data