How does overparametrization affect performance on minority groups?

Author name not available (Why is that?)

Publication date: 7 June 2022

Abstract: The benefits of overparameterization for the overall performance of modern machine learning (ML) models are well known. However, the effect of overparameterization at a more granular level of data subgroups is less understood. Recent empirical studies demonstrate encouraging results: (i) when groups are not known, overparameterized models trained with empirical risk minimization (ERM) perform better on minority groups; (ii) when groups are known, ERM on data subsampled to equalize group sizes yields state-of-the-art worst-group-accuracy in the overparameterized regime. In this paper, we complement these empirical studies with a theoretical investigation of the risk of overparameterized random feature models on minority groups. In a setting in which the regression functions for the majority and minority groups are different, we show that overparameterization always improves minority group performance.

Has companion code repository: https://github.com/smaityumich/overparameterization

This page was built for publication: How does overparametrization affect performance on minority groups?

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6401387)