PAC-Bayesian Contrastive Unsupervised Representation Learning
From MaRDI portal
Publication:6326927
arXiv1910.04464MaRDI QIDQ6326927
Author name not available (Why is that?)
Publication date: 10 October 2019
Abstract: Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2019) provide the first generalisation bounds for CURL, relying on a Rademacher complexity. We extend their framework to the flexible PAC-Bayes setting, allowing us to deal with the non-iid setting. We present PAC-Bayesian generalisation bounds for CURL, which are then used to derive a new representation learning algorithm. Numerical experiments on real-life datasets illustrate that our algorithm achieves competitive accuracy, and yields non-vacuous generalisation bounds.
Has companion code repository: https://github.com/nzw0301/pb-contrastive
This page was built for publication: PAC-Bayesian Contrastive Unsupervised Representation Learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6326927)
